-- Logs begin at Wed 2022-09-07 15:16:10 UTC, end at Wed 2022-09-07 16:19:50 UTC. -- Sep 07 15:41:24.600660 dv-cinder-853315-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 07 15:41:24.608776 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 7 15:41:24 2022] *** Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: nodename: dv-cinder-853315-8-iscsi Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: machine: x86_64 Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: clock source: unix Sep 07 15:41:24.610799 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: pcre jit disabled Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: detected number of CPU cores: 4 Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: current working directory: / Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: detected binary path: /usr/bin/uwsgi-core Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: your processes number limit is 47680 Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: your memory page size is 4096 bytes Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: detected max file descriptor number: 2048 Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: lock engine: pthread robust mutexes Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: thunder lock: enabled Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 07 15:41:24.611263 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 07 15:41:24.651385 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: Python main interpreter initialized at 0x561ea0d7d9c0 Sep 07 15:41:24.651385 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: python threads support enabled Sep 07 15:41:24.651385 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: your server socket listen backlog is limited to 100 connections Sep 07 15:41:24.651385 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: your mercy for graceful operations on workers is 90 seconds Sep 07 15:41:24.651812 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: mapped 403077 bytes (393 KB) for 2 cores Sep 07 15:41:24.651894 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: *** Operational MODE: preforking *** Sep 07 15:41:24.652152 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: *** uWSGI is running in multiple interpreter mode *** Sep 07 15:41:24.652152 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: spawned uWSGI master process (pid: 81445) Sep 07 15:41:24.652351 dv-cinder-853315-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 07 15:41:24.655384 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: spawned uWSGI worker 1 (pid: 81458, cores: 1) Sep 07 15:41:24.655384 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: spawned uWSGI worker 2 (pid: 81459, cores: 1) Sep 07 15:41:24.655384 dv-cinder-853315-8-iscsi devstack@keystone.service[81445]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 07 15:41:28.930675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 15:41:28.930675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: from cryptography.utils import int_from_bytes Sep 07 15:41:28.931462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 15:41:28.931462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: from cryptography.utils import int_from_bytes Sep 07 15:41:28.935807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 15:41:28.935807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: from cryptography.utils import int_from_bytes Sep 07 15:41:28.936262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 07 15:41:28.936262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: from cryptography.utils import int_from_bytes Sep 07 15:41:28.981347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 15:41:28.981537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 15:41:28.982265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] command line args: [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 15:41:28.982547 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 15:41:28.982728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ================================================================================ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 15:41:28.982986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] admin_token = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.983236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_dir = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.983426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.983647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] config_source = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.983728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 07 15:41:28.983830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] debug = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.983909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 07 15:41:28.984029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.984294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] command line args: [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 07 15:41:28.984363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 07 15:41:28.984421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.984550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ================================================================================ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 07 15:41:28.984638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.984774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] admin_token = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.984885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_dir = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] config_source = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_config_append = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] debug = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_dir = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.985985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.986978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_config_append = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_dir = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.987947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.988937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989054 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.989999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] publish_errors = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.990918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] standard_threads = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] publish_errors = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991609 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.991869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_journal = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_json = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_stderr = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.992897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] use_syslog = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] standard_threads = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.993527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.993753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.993986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_journal = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.993986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_json = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.994203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.994321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_stderr = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.994434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.994553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] use_syslog = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.994669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.994787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 07 15:41:28.994900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.995902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.996955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.997068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.997180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.997302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.998939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:28.999945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.000971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.001886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.002932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.003887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.004817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.005948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.006965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.007900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.008995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.009934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.010893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.011953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.012996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.013922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.014964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.015950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.016926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.017994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.018900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.external = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.019998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.password = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.external = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] auth.token = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.020944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.password = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.021885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] auth.token = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.022934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.023929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.024979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.025955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.026945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.027921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.028986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.029927 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.030935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.031937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.032997 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.033935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.034991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.035952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.036896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.037945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.038894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.039952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.040881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.041941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.042966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.043940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.044055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.044175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.044824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.044940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.045976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.046908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.047977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.048903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049586 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.049937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.050988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.051979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.052924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.053921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.054972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.055919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.056984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.057898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.058891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.059890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.060504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.060624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.060741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.060741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.060967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.061925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.062957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.063914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.064955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.065904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.066976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.067939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.068957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.069885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81459) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 15:41:29.070798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 07 15:41:29.070798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81458) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 07 15:41:29.097169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.097319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.097424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.097506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.097620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.097708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.106147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.106322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.135037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.135262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.138789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.139020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.190142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.190283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.239285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.239514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.244642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.244884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.394598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.394820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.394930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.395938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.418808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.419935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.420049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.420183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.420266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.420405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.420625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.431326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.431850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.444417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.444615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.444830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.444988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81458) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.445868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.446082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.446295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81459) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 07 15:41:29.462321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.462540 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 15:41:29.462875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.463039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.463128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 15:41:29.463253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 15:41:29.463337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.463611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.463695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.463821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 07 15:41:29.463916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.464098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.464179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.464364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.464626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.464885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.474500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.474600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.474872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.474955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.475146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.475251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.475428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.475517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.483225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.484784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.486182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.486473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.486758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.487045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.487331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.487624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.487901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.488176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.488256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.488449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.488532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.488726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.488816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.488997 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.489078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.489280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.489547 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.489826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.490106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.490380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.490655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.490924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.512185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.513682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.513968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.514049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.515471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.515785 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.518679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.518966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.519253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.520449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.520744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.521015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.525646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.525936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.527331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.527644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.531683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.531979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.532267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.532556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.532837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.533115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.533323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.533407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.533621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.533706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.533908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.534194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.534470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.534747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.535018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.535286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.549682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.549964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.551323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.551631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.553851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.555467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.557126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.557427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.558698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.558996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.562196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.563770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.565437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.566882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.568693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.568999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.569289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.569589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.569872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.570171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.570263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.570526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.570813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.571106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.571389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.571712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.586014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.586310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.586581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.586849 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.587394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.587707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.587975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.588239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.593816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.595094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.595367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.596659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.596924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.597204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.598185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.598459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.599912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.600200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.600485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.600780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.601172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.601267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.601506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.601797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.602078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.602460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.613145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.613434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.613722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.614008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.614378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.614475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.614657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.614739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.614935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.615018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.615219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.615639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.615932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.616214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.630389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.631572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.633327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.634460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.636248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.637396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.637707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.639389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.639722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.640007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.640402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.640993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.641276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.641672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.646773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.648025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.648299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.649488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.651183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.651611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.651901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.652181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.652401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.652799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.653091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.653372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.660057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.660363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.660648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.661199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.661492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.661791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.667554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81459) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.667853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.668135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.668411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.668621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81458) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 07 15:41:29.668706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.668907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.669000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.669713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.669804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.669999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.670893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81459) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.671926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.672195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.672460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.672742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81458) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 07 15:41:29.706056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561ea0d7d9c0 pid: 81459 (default app) Sep 07 15:41:29.706899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x561ea0d7d9c0 pid: 81458 (default app) Sep 07 15:41:29.709286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf29ac-08b2-4f54-a5db-06ebd0a030e0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:29.709574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf29ac-08b2-4f54-a5db-06ebd0a030e0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:29.709851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf29ac-08b2-4f54-a5db-06ebd0a030e0 None None] PATH_INFO: `/v3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:29.714034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Wed Sep 7 15:41:29 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:36.402331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:36.402832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:36.402942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:36.407813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:36.413164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:36.413647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:36.413647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:36.487876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: INFO dbcounter [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] Registered counter for database keystone Sep 07 15:41:36.507372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG oslo_db.sqlalchemy.engines [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81459) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 15:41:36.513725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writer thread running {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 15:41:36.693036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG passlib.handlers.bcrypt [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81459) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 15:41:36.693337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG passlib.handlers.bcrypt [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81459) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 15:41:36.751284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:36.752930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:36.878887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 07 15:41:37.377527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:37.447020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: INFO dbcounter [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] Registered counter for database keystone Sep 07 15:41:37.468608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG oslo_db.sqlalchemy.engines [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81458) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 07 15:41:37.474809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writer thread running {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 07 15:41:37.607115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d431b-bd4a-4ed0-a2e5-4f211c21c29f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:37.611414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:37.612075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:37.612361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:37.612637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:37.620806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:37.671666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 15:41:37.672014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999a60f-34c1-4a67-9a90-f1c920f60e00 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:37.681284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Sep 7 15:41:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 15:41:37.688158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:37.703043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c7307f-3221-490f-acd9-6c490e275b03 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:37.707318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:37.708444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:37.708726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:37.708995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] PATH_INFO: `/v3/projects/4102cbba3e844357b14c2a5139b42e69` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:37.710531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] RBAC: Authorizing `identity:get_project(project_id=4102cbba3e844357b14c2a5139b42e69)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:37.764198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 07 15:41:37.764540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bffad-5969-441e-a60b-7efe584e24c7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:37.769171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Sep 7 15:41:37 2022] GET /identity/v3/projects/4102cbba3e844357b14c2a5139b42e69 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:39.877675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e49643-297a-4342-990d-cd287c5710c4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:39.878201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e49643-297a-4342-990d-cd287c5710c4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:39.878201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e49643-297a-4342-990d-cd287c5710c4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:39.882343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:39.887736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:39.888130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:39.888293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:39.908699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:39.910211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:39.917922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 07 15:41:40.191354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e49643-297a-4342-990d-cd287c5710c4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:40.203504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e49643-297a-4342-990d-cd287c5710c4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:40.207828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86619-ecc6-40ff-b2b2-b14560a532e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:40.208428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86619-ecc6-40ff-b2b2-b14560a532e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:40.208709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86619-ecc6-40ff-b2b2-b14560a532e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:40.208983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86619-ecc6-40ff-b2b2-b14560a532e2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:40.217214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86619-ecc6-40ff-b2b2-b14560a532e2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:40.218984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86619-ecc6-40ff-b2b2-b14560a532e2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:40.227844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Sep 7 15:41:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 15:41:40.233968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:40.242869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c996c-c811-4544-9251-57a728c90218 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:40.247682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d0f1a-3b77-46b1-ba2e-3b3b4485aed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:40.248306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d0f1a-3b77-46b1-ba2e-3b3b4485aed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:40.248586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d0f1a-3b77-46b1-ba2e-3b3b4485aed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:40.248855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d0f1a-3b77-46b1-ba2e-3b3b4485aed5 admin admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:40.250155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d0f1a-3b77-46b1-ba2e-3b3b4485aed5 admin admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:40.252005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d0f1a-3b77-46b1-ba2e-3b3b4485aed5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:40.253503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Wed Sep 7 15:41:40 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:42.473789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353f90-8f5c-46d9-a430-4c0bfe253e26 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:42.474312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353f90-8f5c-46d9-a430-4c0bfe253e26 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:42.474312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d353f90-8f5c-46d9-a430-4c0bfe253e26 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:42.475616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:42.481091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:42.482051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:42.482195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:42.485939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853e08-c63b-4c12-9219-50b6bf8a383a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:42.486227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853e08-c63b-4c12-9219-50b6bf8a383a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:42.486502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da853e08-c63b-4c12-9219-50b6bf8a383a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:42.487740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:41:42.492799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:42.493085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:42.493363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:42.501292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:42.502478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:42.536065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG passlib.handlers.bcrypt [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81458) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 07 15:41:42.536464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG passlib.handlers.bcrypt [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81458) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 07 15:41:42.580543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:42.585251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:42.586846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:42.686940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:41:42.848748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:42.875443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:42.955803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:42.957491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84945704-2d19-4c30-939b-4ff663bef692 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:42.991528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:42.992190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:42.992467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:42.992737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.004043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:43.013254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:43.014399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:41:43.016603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:41:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:43.020131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:43.058045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34c2b9e-2049-4e81-8569-d181d03c78b6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.060864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:43.096109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.096961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.097239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.097509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.101763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:43.120105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dced4dc8-3cf7-412a-b00d-6a28a403dbb2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.154457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.155076 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.155357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.155664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.156332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:43.156635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.158448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.165432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.166949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.170169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:41:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:41:43.175901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:43.272711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 8/16] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:41:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:41:43.278973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:43.281738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:43.284274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c5438-30bb-4e01-9db4-33135e790534 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.318708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.319349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.319668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.319945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.321086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.322543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.324453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:43 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:41:43.330402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:43.344672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:43.377419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d268b898-bec2-4fea-b704-615da380c441 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.411632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.412479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.412759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.413035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.428605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54b420-99be-491b-b8e1-277ce1130722 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.431380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.432866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:41:43.451095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:43.452054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:41:43.452863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:41:43.452863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:41:43.452863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:41:43.452863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:41:43.457974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:43.464651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.465300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.465581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.466398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.467949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.469764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.471824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:41:43.477657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:43.514184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:43.557503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:43.558795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002a21-78b4-4f8c-a87b-f138dbfcb7f6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.577668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05120495-e47e-4f7d-bc24-4fa39bacc8c6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:43.597935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.598558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.598841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.599114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.599788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:43.600090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.601498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e4f30a-a03e-45e3-9662-ad0a12ebb2ec None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.613239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:43.613853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:43.614140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:43.614421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:43.615155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.domain.id=default, user.id=91fe14f693174ef291042aa145475ad4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:43.615473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:43.616950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f3490-780b-47f6-9e69-35f1e0319c3c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:43.619496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:41:43.624306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Sep 7 15:41:43 2022] GET /identity/v3/role_assignments?user.id=91fe14f693174ef291042aa145475ad4&scope.domain.id=default&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:41:46.320836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01231-4480-4157-b073-310d811d1960 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:46.321336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01231-4480-4157-b073-310d811d1960 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:46.321436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e01231-4480-4157-b073-310d811d1960 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:46.322717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:41:46.333067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568178c8-a08b-4c15-a165-0508d44871ef None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:46.333572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568178c8-a08b-4c15-a165-0508d44871ef None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:46.334281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568178c8-a08b-4c15-a165-0508d44871ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:46.370752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-568178c8-a08b-4c15-a165-0508d44871ef None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:46.373141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-568178c8-a08b-4c15-a165-0508d44871ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:46.529391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 12/23] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:46.575692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4f80d-7cc7-40a7-8f0c-ddd815d5c9ce None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:46.576146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4f80d-7cc7-40a7-8f0c-ddd815d5c9ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:46.576545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f4f80d-7cc7-40a7-8f0c-ddd815d5c9ce None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:46.578219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 12/24] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:46.584637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4059b-fa8d-4deb-a01f-ade1c512ea38 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:46.585360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4059b-fa8d-4deb-a01f-ade1c512ea38 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:46.586061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b4059b-fa8d-4deb-a01f-ade1c512ea38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:46.624210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-00b4059b-fa8d-4deb-a01f-ade1c512ea38 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:46.626537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-00b4059b-fa8d-4deb-a01f-ade1c512ea38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:46.783047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:41:47.067478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4f80d-7cc7-40a7-8f0c-ddd815d5c9ce None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:47.086275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0619381-73cb-4f90-b5f7-38294a0b99a0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.087157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0619381-73cb-4f90-b5f7-38294a0b99a0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.087972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0619381-73cb-4f90-b5f7-38294a0b99a0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.091272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 14/26] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:41:47.116957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96e45a-3905-4504-bd6e-1fda6d97b0ea None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.117792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96e45a-3905-4504-bd6e-1fda6d97b0ea None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.118537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc96e45a-3905-4504-bd6e-1fda6d97b0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.142603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:47.180827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fc96e45a-3905-4504-bd6e-1fda6d97b0ea None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:47.182763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fc96e45a-3905-4504-bd6e-1fda6d97b0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:47.333067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 15/27] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:41:47.334414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794da1dc-196f-4a1e-87e0-d8509926afef None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.335287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794da1dc-196f-4a1e-87e0-d8509926afef None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.335287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794da1dc-196f-4a1e-87e0-d8509926afef None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.336486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 16/28] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:47.337105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794da1dc-196f-4a1e-87e0-d8509926afef None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:47.359476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f4f80d-7cc7-40a7-8f0c-ddd815d5c9ce None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:47.389393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:47.426307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:47.426946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.427248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.427502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.444853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:47.446331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-95cfa4cc-ac43-4729-a31e-c7b5e618cd4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:41:47.473702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:47.475386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:41:47.477035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:41:47.477035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:41:47.477035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:41:47.479475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 13/29] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:47.483066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.483333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.486923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.519782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:47.521007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:47.526622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794da1dc-196f-4a1e-87e0-d8509926afef None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:47.604518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c97b9c-0ab0-4b74-b606-a363bcbbe625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:47.605305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97b9c-0ab0-4b74-b606-a363bcbbe625 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.605591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97b9c-0ab0-4b74-b606-a363bcbbe625 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.605864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c97b9c-0ab0-4b74-b606-a363bcbbe625 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.606610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c97b9c-0ab0-4b74-b606-a363bcbbe625 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:47.608045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c97b9c-0ab0-4b74-b606-a363bcbbe625 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:47.612046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:47.667766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 14/30] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:47.668275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:47.779305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 17/31] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 15:41:47 2022] POST /identity/v3/roles => generated 225 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:41:47.780796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5b7b-4e1e-459a-b264-4701f76782a2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.781089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5b7b-4e1e-459a-b264-4701f76782a2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.781347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e5b7b-4e1e-459a-b264-4701f76782a2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.782556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 18/32] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:47.788254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.788595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.788947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.796676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:47.800980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9e4b87-e03c-41c9-990a-070e51115500 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:47.817232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:47.818765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:47.841244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:47.841903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:47.842713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:47.842713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:47.843701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:47.843701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:47.844879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:47.859007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 15/33] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:41:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:47.868954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:47.929156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 19/34] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:47.970897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.049470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fabcd59-151c-4aca-94a8-b4fc159fcdbf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.094416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.095056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.095337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.095634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.097451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:48.099790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:48.209865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024a1f6-a432-4efd-82c8-9428feb12508 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.263963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:48.296112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b911f4ef-a370-493e-9e83-4bdd1eedff00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.296770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911f4ef-a370-493e-9e83-4bdd1eedff00 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.297061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 16/35] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:41:47 2022] POST /identity/v3/projects => generated 301 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:41:48.297160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911f4ef-a370-493e-9e83-4bdd1eedff00 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.297385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b911f4ef-a370-493e-9e83-4bdd1eedff00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.298137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.319782 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b911f4ef-a370-493e-9e83-4bdd1eedff00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:48.322236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b911f4ef-a370-493e-9e83-4bdd1eedff00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:48.325142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:48.326034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:41:48.326795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 20/36] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:41:48.328127 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb397b-6f63-4a91-be28-0d326731406d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.328425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb397b-6f63-4a91-be28-0d326731406d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.328712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb397b-6f63-4a91-be28-0d326731406d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.330422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 21/37] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:48.336692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.337253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.337253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.368417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:48.370401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:48.439102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759f08f-a3cb-42c8-9ebe-17fc92be6383 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.463968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 22/38] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:48.464853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.496240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.496458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.496674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.497454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.497638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:48.499189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:48.517061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 17/39] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 15:41:48 2022] POST /identity/v3/roles => generated 219 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:41:48.517995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.607422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5164ca5d-e53b-4b48-98f8-2577b9955d58 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.608426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:48.649379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841463f4-57f4-4df5-873e-513e4b26e6aa None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.692624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.693299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.693590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.693868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.695282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:48.696992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:48.699732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 18/40] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:48.709380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.752672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:48.777802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.777802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.777802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.777802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.778901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:48.779382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:48.781075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:48.792102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 23/41] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:41:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:48.792908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.841127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ad8be-dc15-49bc-b7bc-2b92cc549d16 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.874285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.874934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.875218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.875492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.876173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:48.877628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:48.889016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:48.897948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499a777-7add-425b-bdd4-1cd75bb1a288 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:48.906231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 19/42] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:41:48 2022] POST /identity/v3/projects => generated 312 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:41:48.907105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:48.936277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:48.936975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:48.937259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:48.937564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:48.938254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:48.939724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:48.959666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 24/43] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 15:41:48 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:41:49.044022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1e947-5f31-4616-9232-8949c13521de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:49.084211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:49.084955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:49.085243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:49.085521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:49.087022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:49.088549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:49.090381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 20/44] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:48 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:49.096242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:49.139000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:49.211096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102f16e-7f06-4710-be09-851d3912c412 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:49.245686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b5735-7c31-4725-9337-8f920e75f8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:49.246331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5735-7c31-4725-9337-8f920e75f8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:49.246614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5735-7c31-4725-9337-8f920e75f8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:49.247399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5735-7c31-4725-9337-8f920e75f8d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:49.248534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5735-7c31-4725-9337-8f920e75f8d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:49.249985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5735-7c31-4725-9337-8f920e75f8d6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:49.251595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 25/45] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:49.257140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:49.314587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:49.356972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0898c30-961e-4d79-8ac9-fb8e94c315df None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:49.389565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eb504d-12e0-4c5e-9a6c-193fca19a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:49.390231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb504d-12e0-4c5e-9a6c-193fca19a1a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:49.390515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb504d-12e0-4c5e-9a6c-193fca19a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:49.390798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb504d-12e0-4c5e-9a6c-193fca19a1a9 None admin] PATH_INFO: `/v3/domains/default/users/91fe14f693174ef291042aa145475ad4/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:49.392744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb504d-12e0-4c5e-9a6c-193fca19a1a9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=91fe14f693174ef291042aa145475ad4, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:49.394180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eb504d-12e0-4c5e-9a6c-193fca19a1a9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:49.409042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 21/46] 10.222.0.17 () {58 vars in 1361 bytes} [Wed Sep 7 15:41:49 2022] PUT /identity/v3/domains/default/users/91fe14f693174ef291042aa145475ad4/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:41:51.098070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352584f-88ea-4b22-98a2-3709823c4de8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:51.098629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352584f-88ea-4b22-98a2-3709823c4de8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:51.098629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f352584f-88ea-4b22-98a2-3709823c4de8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:51.100107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 26/47] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:51.105625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22758-e7bb-4982-ae22-d2cb77191585 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:51.106521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22758-e7bb-4982-ae22-d2cb77191585 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:51.106714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b22758-e7bb-4982-ae22-d2cb77191585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:51.125585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b9b22758-e7bb-4982-ae22-d2cb77191585 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:51.127110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b9b22758-e7bb-4982-ae22-d2cb77191585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:51.195909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 22/48] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:51.462204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352584f-88ea-4b22-98a2-3709823c4de8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:51.532488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723c72-21cc-421b-9a15-3322eb0ba83f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:51.533126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723c72-21cc-421b-9a15-3322eb0ba83f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:51.533533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1723c72-21cc-421b-9a15-3322eb0ba83f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:51.534984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 23/49] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:51.540383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:51.540671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:51.540950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:51.559299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:51.560538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:51.565107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f352584f-88ea-4b22-98a2-3709823c4de8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:51.567424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:51.600068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:51.600698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:51.600983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:51.601262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:51.602394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:51.603861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:51.603973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:51.605739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 27/50] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:51.611249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:51.628972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 24/51] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:41:51.707976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa553447-f2ac-46d5-8c22-caf60176ab85 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:51.740851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:51.741390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:51.741672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:51.741946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:51.742590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:51.744072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:51.771869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 28/52] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Sep 7 15:41:51 2022] POST /identity/v3/projects => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:41:51.895968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:51.996822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685cddc5-bc82-4214-9a38-dc1f48698b23 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:52.030587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:52.031228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:52.031510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:52.031814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:52.042651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:52.049626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:52.050649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:41:52.051776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 25/53] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:41:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:52.056509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:52.082510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:52.157584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e8d529-6ea6-4b85-8456-058dd6f37bc9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:52.192661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:52.193276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:52.193561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:52.193839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:52.194506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:52.194817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:52.196245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:52.204747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 29/54] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:41:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:52.210571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:52.251431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:52.416579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8aea0-5512-4b7e-955e-9ec15f27b34b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:52.449517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:52.450696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:52.450977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:52.451254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:52.452420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:52.453883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:52.455730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 26/55] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:52 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:52.460920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:52.545149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:52.559908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e77766-818d-4bf7-9760-c268aed6a68a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:52.593183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4198a-2623-4d7f-a352-e283afb0b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:52.593821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4198a-2623-4d7f-a352-e283afb0b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:52.594110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4198a-2623-4d7f-a352-e283afb0b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:52.594392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4198a-2623-4d7f-a352-e283afb0b6f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:52.595510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4198a-2623-4d7f-a352-e283afb0b6f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:52.597017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4198a-2623-4d7f-a352-e283afb0b6f5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:52.598596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 30/56] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:52.604352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:52.704856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034998e-f744-4fef-a188-fd641bb71e90 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:52.711529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:52.737813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:52.738423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:52.738710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:52.738986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:52.739742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.domain.id=default, user.id=91fe14f693174ef291042aa145475ad4) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:52.740056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:52.741498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deff61c-22a1-4a7f-8db7-841ab0b7a17c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:52.749798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 27/57] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Sep 7 15:41:52 2022] GET /identity/v3/role_assignments?user.id=91fe14f693174ef291042aa145475ad4&scope.domain.id=default&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:53.919168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3521e0a-4334-4e60-9e80-f0ea987eb55e None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:53.919706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3521e0a-4334-4e60-9e80-f0ea987eb55e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:53.919812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3521e0a-4334-4e60-9e80-f0ea987eb55e None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:53.921198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:53.926534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:53.926938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:53.927094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:53.944893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:53.946108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:54.010590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:54.280748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3521e0a-4334-4e60-9e80-f0ea987eb55e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:54.380097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3521e0a-4334-4e60-9e80-f0ea987eb55e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:54.412533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a8ba2-83bb-44ae-bc15-84fc3affe33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:54.413152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a8ba2-83bb-44ae-bc15-84fc3affe33b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:54.413433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a8ba2-83bb-44ae-bc15-84fc3affe33b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:54.413708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a8ba2-83bb-44ae-bc15-84fc3affe33b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:54.414796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a8ba2-83bb-44ae-bc15-84fc3affe33b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:54.416287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88a8ba2-83bb-44ae-bc15-84fc3affe33b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:54.417809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 32/60] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:54.423733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:54.516946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:54.519849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603615a-76a6-47c7-8d4b-ff21c4f9a5d7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:54.554174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05385bf-3819-4e2a-a1ac-752ade179e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:54.554728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05385bf-3819-4e2a-a1ac-752ade179e63 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:54.555013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05385bf-3819-4e2a-a1ac-752ade179e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:54.555291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05385bf-3819-4e2a-a1ac-752ade179e63 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:54.555977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05385bf-3819-4e2a-a1ac-752ade179e63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:54.557434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05385bf-3819-4e2a-a1ac-752ade179e63 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:54.582355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 29/61] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:41:54 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:41:57.852814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99e67-b0aa-4ce5-88db-4c947b7d57a6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:57.855584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99e67-b0aa-4ce5-88db-4c947b7d57a6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:57.855584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b99e67-b0aa-4ce5-88db-4c947b7d57a6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:57.855794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 33/62] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:57.861945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7bc9b-dc7f-4c57-84d8-b9b6b28450d6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:57.862771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7bc9b-dc7f-4c57-84d8-b9b6b28450d6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:57.863229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7bc9b-dc7f-4c57-84d8-b9b6b28450d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:57.889538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dbd7bc9b-dc7f-4c57-84d8-b9b6b28450d6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:57.891060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dbd7bc9b-dc7f-4c57-84d8-b9b6b28450d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:57.978190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 30/63] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:58.086561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324a16-17ad-47b5-958c-7fccb2481f11 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.087198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324a16-17ad-47b5-958c-7fccb2481f11 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.087716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e324a16-17ad-47b5-958c-7fccb2481f11 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.089733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 34/64] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:58.096528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.097159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.097634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.118011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:58.119517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:58.188157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8efff5-82c3-4bf9-86ba-1e3748ee3c23 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.188441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8efff5-82c3-4bf9-86ba-1e3748ee3c23 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.188881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8efff5-82c3-4bf9-86ba-1e3748ee3c23 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.190683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 35/65] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:58.196190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354d641-4763-4327-8d5c-6861051e00d5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.196756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354d641-4763-4327-8d5c-6861051e00d5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.197231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354d641-4763-4327-8d5c-6861051e00d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.227329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 31/66] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:58.239874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4354d641-4763-4327-8d5c-6861051e00d5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:58.241103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4354d641-4763-4327-8d5c-6861051e00d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:58.265008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:58.337061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 36/67] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:41:58.339979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:58.380254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:58.505963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc481da-e073-4386-bfda-4e11521f3798 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:58.511878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277fa1-05d2-4070-9bd0-80969ff2dcb2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.512941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277fa1-05d2-4070-9bd0-80969ff2dcb2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.513418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e277fa1-05d2-4070-9bd0-80969ff2dcb2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.517698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 37/68] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:58.526881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ea4af-1fcc-409b-a095-8ee685011377 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.527453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ea4af-1fcc-409b-a095-8ee685011377 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.527952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ea4af-1fcc-409b-a095-8ee685011377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.547888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:58.548530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.548748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-678ea4af-1fcc-409b-a095-8ee685011377 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:58.548870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.549081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.550467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-678ea4af-1fcc-409b-a095-8ee685011377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:58.560064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:58.565293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:58.566152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:41:58.566935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 32/69] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:41:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:58.567599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:58.630679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 38/70] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:41:58.631992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a14a-3eb3-487a-b7c6-7b29c7146b52 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.632284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a14a-3eb3-487a-b7c6-7b29c7146b52 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.632557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc02a14a-3eb3-487a-b7c6-7b29c7146b52 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.633985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 39/71] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:41:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:41:58.634907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a14a-3eb3-487a-b7c6-7b29c7146b52 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:58.652016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:58.681191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97d7dfc-eaee-4b4d-950b-0cb3978dbdb6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:58.718453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:58.719210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.719495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.719879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.720166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:58.736551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:58.740673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc02a14a-3eb3-487a-b7c6-7b29c7146b52 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:58.743120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:58.744179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:41:58.744973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 33/72] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:41:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:58.745598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:58.782051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:58.782678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.782966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.783250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.783959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:58.784272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:58.785679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a63ea-a103-42c8-b0b0-9f1d25d6c672 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:58.794920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 40/73] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:41:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:58.796130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.796424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.796707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.814274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:41:58.815475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:41:58.853718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c92ef9-49bc-49fd-af13-1d1ad79f8659 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:58.885410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 41/74] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:41:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:41:58.886170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:58.906345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:58.907027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:58.907533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:58.907846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:58.912662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:58.922222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:58.927134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:58.928002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:41:58.928781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 34/75] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:41:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:58.929427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:58.971734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:58.999682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37be4703-9821-453b-9c56-edf8b27497cb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.034368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.034991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.035291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.035579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.036261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:59.036575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.037970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.038087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f023c9-2fc5-41c8-bd89-9ec21871d9a6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.046040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 42/76] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:41:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.046825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.079971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.080607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.080896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.081340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.098002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.099676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.106739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 35/77] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:58 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.106739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.165724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2191-2fc2-4706-a332-99bb5e9b7a2b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.167468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.201251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.201890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.202177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.202458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.205161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.213627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:41:59.215041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c370a-db44-4c0c-94ca-3b7a1a49983c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:41:59.219638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:41:59.220763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:41:59.221574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 43/78] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:41:59.222216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.249287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.249907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.250192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.250467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.251132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:59.251438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.252855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.265118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 36/79] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.268429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.321544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384dbc4-eccc-47d1-ac63-8197eb935350 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.354636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.355557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.356378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.356659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.357810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.359289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.360910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 44/80] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.361674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.370425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c5d31-56a8-4d4a-94ef-c47c0362c414 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.387811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.406420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.407029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.407310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.407607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.422733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.461847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab571fbf-e859-48cb-9b9f-0a675ac07a6e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.475800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.477540 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.494910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.495584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.495851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.496129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.497786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.499240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.500845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 45/81] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.501610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.586639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 37/82] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:41:59.587498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.608251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093ccae-6fd0-4f14-82f2-79975bdff659 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.609179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.636725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.641665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.642277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.642562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.642840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.643500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:41:59.643863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.645250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.653548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 46/83] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.654323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.696361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e447c75b-1680-4b9e-b30c-4838bba43567 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.731217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.731880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.732163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.732442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.733589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.735024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.736617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 38/84] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:41:59.737352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.753117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cce80b-c262-435d-8e95-42463546ffa4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.786303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.786917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.787204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.787482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.819473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.835143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1ccca-da42-4ff8-bcda-c940462b58d2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:41:59.848503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:41:59.859564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.861595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.875323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:41:59.875985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:41:59.876271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:41:59.876554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:41:59.877205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:41:59.878649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:41:59.900077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 39/85] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:41:59 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:41:59.900896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:41:59.979706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 47/86] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:41:59.982024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.022703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59f8f3-6e57-4243-b4dc-351dbf72793e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.061335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.062273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.062556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.062832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.064053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.065612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:00.072073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 40/87] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:00.072073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.083317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:00.092699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:00.100704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413eeec0-63db-4df1-af1f-e8ba60fe1464 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.147951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.148596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.148879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.149152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.150414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.151900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:00.153778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 48/88] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:41:59 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:00.154291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.201102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9bf03-4bd7-4816-89d8-454f04a71cc2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.248952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.250207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.250869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.250869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.296672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33939aef-10cf-4b32-acfd-957362df91d6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.335388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.337299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:00.342275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.342933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.343218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.343492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.344679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.346130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:00.347839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 49/89] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:00 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:00.349017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.357596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:00.393868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:00.471671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 41/90] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:00.476917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.478203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bdba-ffd4-44fa-bba9-b1a3864bae06 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.517112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.517446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.517735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.518021 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.518773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=597cf93046794f75bf6b7a42a09fb48c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:00.519095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.520637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:00.531574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 50/91] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:00 2022] GET /identity/v3/role_assignments?user.id=597cf93046794f75bf6b7a42a09fb48c&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:00.531574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.619440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7245c3-764e-48b5-ad88-8cf9c124a8cf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.631767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:00.664761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.665734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.666234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.666685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.752130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7effec-2c65-47b0-8462-5f4fa7373796 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:00.753973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:00.791815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:00.792791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:00.793807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:00.794439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:00.795422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=91fe14f693174ef291042aa145475ad4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:00.795968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.797756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:00.808108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 51/92] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:00 2022] GET /identity/v3/role_assignments?user.id=91fe14f693174ef291042aa145475ad4&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:00.809291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:00.820555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:00.823057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:01.027203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd089124-9521-4d75-9752-46f66ab2d310 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:01.063420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 42/93] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:01.069279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:01.084749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:01.100309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:01.101492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:01.102089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:01.102621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] PATH_INFO: `/v3/projects/612572751f1544f9ae840f085e09ad0d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:01.104128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] RBAC: Authorizing `identity:get_project(project_id=612572751f1544f9ae840f085e09ad0d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:01.106259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:01.108141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 52/94] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:00 2022] GET /identity/v3/projects/612572751f1544f9ae840f085e09ad0d => generated 312 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:01.114785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:01.183172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:01.282307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804418d-8631-4233-ad68-243aaac93f0b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:01.294334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be17b9d-25d8-4501-a899-9f7a50d7019c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:01.341723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:01.342722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:01.343016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:01.343293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:01.344066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=612572751f1544f9ae840f085e09ad0d, user.id=597cf93046794f75bf6b7a42a09fb48c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:01.344538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:01.346020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:01.358401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 53/95] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:01 2022] GET /identity/v3/role_assignments?user.id=597cf93046794f75bf6b7a42a09fb48c&scope.project.id=612572751f1544f9ae840f085e09ad0d&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:01.373791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52df796-c463-4d55-853a-41e8f15dd7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:01.374462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52df796-c463-4d55-853a-41e8f15dd7ad None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:01.374742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52df796-c463-4d55-853a-41e8f15dd7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:01.375014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52df796-c463-4d55-853a-41e8f15dd7ad None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:01.377819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52df796-c463-4d55-853a-41e8f15dd7ad None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:01.379297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52df796-c463-4d55-853a-41e8f15dd7ad None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:01.381262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 43/96] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:01 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:01.388029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:01.526716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:01.537193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a42716-8d7b-4d28-9de9-261ea5c4c332 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:01.577463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:01.578089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:01.578372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:01.578644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:01.579445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] RBAC: Adding query filter params (role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=597cf93046794f75bf6b7a42a09fb48c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:01.579787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:01.581258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24122050-af9e-4da3-ad68-cb8340399b6a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:01.587595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 54/97] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:01 2022] GET /identity/v3/role_assignments?user.id=597cf93046794f75bf6b7a42a09fb48c&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:02.632793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb00e3-63d4-4e9f-a959-b6622aa54e79 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:02.633327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb00e3-63d4-4e9f-a959-b6622aa54e79 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:02.633327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb00e3-63d4-4e9f-a959-b6622aa54e79 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:02.634936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 44/98] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:02.640443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de1d8-d953-4992-a5b2-88509dc70ab2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:02.640832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de1d8-d953-4992-a5b2-88509dc70ab2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:02.641017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3de1d8-d953-4992-a5b2-88509dc70ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:02.664297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7b3de1d8-d953-4992-a5b2-88509dc70ab2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:02.665503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7b3de1d8-d953-4992-a5b2-88509dc70ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:02.748774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 55/99] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:03.034865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaabd-0524-4a2d-9143-9fdf374f4a67 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.035255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaabd-0524-4a2d-9143-9fdf374f4a67 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.035574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eeaabd-0524-4a2d-9143-9fdf374f4a67 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.037011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 45/100] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:03.042333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.042670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.042945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.067717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:03.068906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:03.147234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaabd-0524-4a2d-9143-9fdf374f4a67 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:03.163060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 56/101] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:03.273920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:03.310621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eeaabd-0524-4a2d-9143-9fdf374f4a67 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:03.357612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60420bb7-2891-4a82-bd62-97d066ffefdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:03.358261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420bb7-2891-4a82-bd62-97d066ffefdf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.358545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420bb7-2891-4a82-bd62-97d066ffefdf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.358822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60420bb7-2891-4a82-bd62-97d066ffefdf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.369906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60420bb7-2891-4a82-bd62-97d066ffefdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-60420bb7-2891-4a82-bd62-97d066ffefdf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:03.375356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:03.376352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:03.377691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 46/102] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:03.384639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:03.409926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:03.469611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddc38-d6a2-4a48-905a-a20e0a52ab9b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.469943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddc38-d6a2-4a48-905a-a20e0a52ab9b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.470227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80ddc38-d6a2-4a48-905a-a20e0a52ab9b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.471772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 47/103] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:03.476407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.476697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.476983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.508638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:03.509871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:03.550580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4a357-f0ea-4f5b-9427-bc232c671ed6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:03.615788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:03.616332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.616617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.616894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.617569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:03.617879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:03.619264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:03.625471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 48/104] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:03.628322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 57/105] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:03.633783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:03.723732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:03.731930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:03.804452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:03.810165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea8c200-9984-44dd-9432-2a8ce9930223 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:03.830480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9dc38-245d-40e1-92a1-79804de5e140 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:03.867864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:03.868443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.868728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.869002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.870162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:03.871713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:03.873472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 58/106] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:03.880674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:03.883253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:03.884189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:03.884664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:03.885127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] PATH_INFO: `/v3/users/54b881f9b5fc4362a3e31f2060bf8350` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:03.904490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] RBAC: Authorizing `identity:get_user(user_id=54b881f9b5fc4362a3e31f2060bf8350)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:03.907378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:03.914597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 49/107] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:03 2022] GET /identity/v3/users/54b881f9b5fc4362a3e31f2060bf8350 => generated 289 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:03.921415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:04.102583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acb2d2-48c7-4d1e-a7e0-7e995ea4a066 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:04.113810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d402e-483f-4b5a-b960-7409e5672516 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:04.143744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:04.176080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675ce82e-0ba0-4b33-bc51-ab6a0a52451d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:04.176840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce82e-0ba0-4b33-bc51-ab6a0a52451d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.177133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce82e-0ba0-4b33-bc51-ab6a0a52451d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.177416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675ce82e-0ba0-4b33-bc51-ab6a0a52451d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.179428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:04.180382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.181227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.181699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.182576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:04.186586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:04.226532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:04.244117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 59/108] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:42:03 2022] POST /identity/v3/projects => generated 302 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:42:04.245280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:04.325075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce82e-0ba0-4b33-bc51-ab6a0a52451d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:04.328216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675ce82e-0ba0-4b33-bc51-ab6a0a52451d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:04.416994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2c1d7-aafb-4e89-ad18-409e6a03fbbc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:04.499446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:04.500181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.500461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.500730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.517657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:04.519720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 50/109] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:04.521201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ef3a-0f38-4b4c-a575-2022b7787571 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.521692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ef3a-0f38-4b4c-a575-2022b7787571 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.527686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93ef3a-0f38-4b4c-a575-2022b7787571 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:04.527869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:04.528679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 60/110] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:04.529457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:04.530142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 51/111] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:04.535770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.536096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.536344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.568674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:04.569966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:04.593282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:04.618709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:04.657861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 52/112] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:04.658798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:04.680171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c408d29-622c-438d-a35b-e865d8d99aa3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:04.718695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596f62ff-f642-43c8-aa3b-8af184f81e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:04.719752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f62ff-f642-43c8-aa3b-8af184f81e9c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.720236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f62ff-f642-43c8-aa3b-8af184f81e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.720707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596f62ff-f642-43c8-aa3b-8af184f81e9c None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.722122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f62ff-f642-43c8-aa3b-8af184f81e9c None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:04.723828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596f62ff-f642-43c8-aa3b-8af184f81e9c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:04.726133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 61/113] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:04 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:04.727530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b198e-430b-4297-802d-8db05930f5f7 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.732095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b198e-430b-4297-802d-8db05930f5f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.732576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3b198e-430b-4297-802d-8db05930f5f7 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.734032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 62/114] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:04.736191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b198e-430b-4297-802d-8db05930f5f7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:04.778210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c9074b-92dd-4fa4-80b1-ee25e48f6b85 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:04.829285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:04.831648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.833104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.833623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.835218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:04.837095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:04.838711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c567c57-db30-4427-9646-673d033124e4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:04.853688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3b198e-430b-4297-802d-8db05930f5f7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:04.858179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 53/115] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:04.866059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ee9b-9357-4e52-8698-d3da522dacb5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.872800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ee9b-9357-4e52-8698-d3da522dacb5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.873410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1ee9b-9357-4e52-8698-d3da522dacb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.911083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f8b1ee9b-9357-4e52-8698-d3da522dacb5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:04.912349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8b1ee9b-9357-4e52-8698-d3da522dacb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:04.917620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:04.927025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:04.927944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:04.928427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:04.928908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:04.929857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] RBAC: Adding query filter params (role.id=1c08be9ad98543729654cd64c1c27930, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=54b881f9b5fc4362a3e31f2060bf8350) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:04.930421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:04.932120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:04.935884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:04.943392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 63/116] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:04 2022] GET /identity/v3/role_assignments?user.id=54b881f9b5fc4362a3e31f2060bf8350&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=1c08be9ad98543729654cd64c1c27930 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:04.949870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.041815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 54/117] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:05.046371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7685b-89a9-425f-a3ec-6fbe30b3a400 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.046660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7685b-89a9-425f-a3ec-6fbe30b3a400 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.046940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf7685b-89a9-425f-a3ec-6fbe30b3a400 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.048719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 55/118] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:05.049332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7685b-89a9-425f-a3ec-6fbe30b3a400 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.141009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfec7db-229a-48ab-aa44-1d63880bbbb8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.204621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b542f-d935-422b-b9fe-10f1ec6e7050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.205653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b542f-d935-422b-b9fe-10f1ec6e7050 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.206165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b542f-d935-422b-b9fe-10f1ec6e7050 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.206634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b542f-d935-422b-b9fe-10f1ec6e7050 None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.212380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b542f-d935-422b-b9fe-10f1ec6e7050 None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:05.214659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b542f-d935-422b-b9fe-10f1ec6e7050 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:05.217212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 64/119] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:04 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:05.218696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.223839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.224360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.245777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:05.247508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:05.268479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf7685b-89a9-425f-a3ec-6fbe30b3a400 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.335253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.336210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.336501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.336783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.339202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:05.352360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:05.359536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:05.360520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:05.361297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:05.363510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 65/120] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:05.363777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 56/121] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:05.364841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.369136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.375402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:05.477018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddb148b-06d2-427d-aafb-8d6c63a78187 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.515241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.516447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.516944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.517490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.531808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:05.537863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:05.538774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:05.539776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:05.541802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 57/122] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:05.542831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.546934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23bfd6-af87-4b27-adc7-3365d10d294e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.584611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.585041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.585334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.585618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.586778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:05.588286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:05.590461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 66/123] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:05.591603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.639820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:05.656770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:05.702282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdb962-0c2f-4a3e-95b3-2d9c7f78bc71 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.704126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad94e56-27c0-4ad3-b2bf-f3ce9952ee8a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.740237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.740810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.741040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.741315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.741981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:05.742289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:05.743970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:05.753045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 67/124] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:05.753397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.754012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.754398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.754682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.755354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:05.755708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:05.757101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:05.758359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.767157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 58/125] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:05.772314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.868007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc656442-5e04-42b4-8b86-7a5907acd324 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.874890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426fd1f-ffd8-43ff-8c9b-987a02ac63e2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:05.885807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:05.902076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:05.910113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.910712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:05.911138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.911353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:05.911753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.911861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:05.912038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.912391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/597cf93046794f75bf6b7a42a09fb48c/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:05.914896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b378b246cf9b49a9b0410534f7cc6d92, user_id=597cf93046794f75bf6b7a42a09fb48c, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:05.916545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:05.924077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:05.929673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:05.930724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 59/126] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:05.931568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:05.942224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 68/127] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:05 2022] PUT /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/597cf93046794f75bf6b7a42a09fb48c/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:05.943393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.073160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7501a9-bb6b-49e0-a10b-ef144efca07e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.076160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7978e9dd-9263-429c-ab0a-3e2957d39da9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.115076 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.116032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.116382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.116726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.118171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.120179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.120887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.121765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.122191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.122367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 69/128] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:06.122860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.123396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.124637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.126258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.131072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 60/129] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:05 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:06.132234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.194017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:06.213276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:06.267016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac020c-16c6-41c8-8059-4b9213f640ea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.276958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ad7f89-c242-48ad-b8b1-31198e7a294e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.303574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.304467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.304890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.305306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.306570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.308202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.310492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 61/130] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:06 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:06.311566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.314882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.316049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.316049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.316273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.316869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:06.317178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.318653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.328586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 70/131] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:06.329458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.417376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f19a8-3011-41bd-af87-ef199f56cebd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.441885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:06.453310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.454160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.454608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.455023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] PATH_INFO: `/v3/projects/612572751f1544f9ae840f085e09ad0d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.456380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] RBAC: Authorizing `identity:get_project(project_id=612572751f1544f9ae840f085e09ad0d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.457979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.460156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 62/132] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:06 2022] GET /identity/v3/projects/612572751f1544f9ae840f085e09ad0d => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:06.461158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.484091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c7c3dd-b166-4f76-90a1-f118aa9320a6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.499995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:06.551772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.552470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.552756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.553036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/91fe14f693174ef291042aa145475ad4/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.557680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] RBAC: Authorizing `identity:create_grant(project_id=b378b246cf9b49a9b0410534f7cc6d92, user_id=91fe14f693174ef291042aa145475ad4, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.559754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.565006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e141a545-dff5-449e-a9ea-3ffbb44e5792 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.593381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 71/133] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:06 2022] PUT /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/91fe14f693174ef291042aa145475ad4/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:06.594120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.600845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.601738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.602188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.602603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.604035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.605992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.607860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 63/134] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:06 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:06.613782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.690927 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:06.727222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b3c408-131f-40ba-bed7-6e41d888b7f1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.767183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.768257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.769908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.770371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.772489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.775811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.778606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 64/135] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:06 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:06.784850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:06.817844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02823bc1-07fb-4f04-ae19-da00d970ddfe None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:06.890783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e9de4c-9448-4c24-aa9c-981d2e757dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:06.891446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9de4c-9448-4c24-aa9c-981d2e757dd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:06.891765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9de4c-9448-4c24-aa9c-981d2e757dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:06.892296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e9de4c-9448-4c24-aa9c-981d2e757dd8 None admin] PATH_INFO: `/v3/projects/612572751f1544f9ae840f085e09ad0d/users/597cf93046794f75bf6b7a42a09fb48c/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:06.900536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9de4c-9448-4c24-aa9c-981d2e757dd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=612572751f1544f9ae840f085e09ad0d, user_id=597cf93046794f75bf6b7a42a09fb48c, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:06.902064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e9de4c-9448-4c24-aa9c-981d2e757dd8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:06.922182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 72/136] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:06 2022] PUT /identity/v3/projects/612572751f1544f9ae840f085e09ad0d/users/597cf93046794f75bf6b7a42a09fb48c/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:06.974868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37df4c63-a88a-499c-a952-6a60ad45db06 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:07.019780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:07.030038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf291d-20d3-4322-af6a-bc9e554e3d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:07.031206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf291d-20d3-4322-af6a-bc9e554e3d95 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:07.031758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf291d-20d3-4322-af6a-bc9e554e3d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:07.032238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf291d-20d3-4322-af6a-bc9e554e3d95 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/597cf93046794f75bf6b7a42a09fb48c/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:07.035328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf291d-20d3-4322-af6a-bc9e554e3d95 None admin] RBAC: Authorizing `identity:create_grant(project_id=b378b246cf9b49a9b0410534f7cc6d92, user_id=597cf93046794f75bf6b7a42a09fb48c, role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:07.037259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf291d-20d3-4322-af6a-bc9e554e3d95 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:07.055263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 65/137] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:06 2022] PUT /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/597cf93046794f75bf6b7a42a09fb48c/roles/0cc328004b4e4f44905d38cb222157a9 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:07.158911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e30a-3064-4b93-ba1d-47799e2c3599 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:07.159310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e30a-3064-4b93-ba1d-47799e2c3599 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:07.159683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423e30a-3064-4b93-ba1d-47799e2c3599 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:07.161116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 73/138] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:07.167059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:07.167466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:07.167836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:07.195072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:07.196654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:07.290810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 66/139] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:07.693631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e30a-3064-4b93-ba1d-47799e2c3599 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:07.741180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:07.837151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423e30a-3064-4b93-ba1d-47799e2c3599 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:07.882540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac6db6-8d1a-4368-9707-db670420336b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:07.883165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6db6-8d1a-4368-9707-db670420336b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:07.883450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6db6-8d1a-4368-9707-db670420336b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:07.883745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac6db6-8d1a-4368-9707-db670420336b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:07.884838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac6db6-8d1a-4368-9707-db670420336b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:07.886383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac6db6-8d1a-4368-9707-db670420336b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:07.887945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 74/140] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:07.893372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:08.015949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f247a-8739-451d-8122-1d06ea2afc08 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:08.047952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:08.055679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9113c-bef5-40ce-a8ed-c2e4e7e0dfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:08.057063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9113c-bef5-40ce-a8ed-c2e4e7e0dfef None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:08.057350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9113c-bef5-40ce-a8ed-c2e4e7e0dfef None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:08.057636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9113c-bef5-40ce-a8ed-c2e4e7e0dfef None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:08.058320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9113c-bef5-40ce-a8ed-c2e4e7e0dfef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:08.060629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9113c-bef5-40ce-a8ed-c2e4e7e0dfef None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:08.085276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 67/141] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:07 2022] POST /identity/v3/users => generated 283 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:08.202984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518492c-a6dd-4986-99da-71f9ca0a76c9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:08.203269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518492c-a6dd-4986-99da-71f9ca0a76c9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:08.203557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518492c-a6dd-4986-99da-71f9ca0a76c9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:08.204943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 75/142] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:08.210451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:08.210730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:08.211007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:08.231217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:08.236548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:08.330198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 68/143] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:08.887742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518492c-a6dd-4986-99da-71f9ca0a76c9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:09.012225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:09.022900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518492c-a6dd-4986-99da-71f9ca0a76c9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:09.063151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:09.063822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.064182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.064464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.078695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:09.083625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:09.084464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:09.085316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 76/144] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:09.092690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:09.272783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ed708-3ccf-47ac-a927-1f2fe4f7034e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:09.276624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:09.333550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:09.334428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.334838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.335237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.336035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:09.336512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:09.338372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1382cb1-489e-4bdb-9549-674e2d6991c8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:09.349754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 69/145] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:09.356114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:09.426825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38969-8582-4c06-9a35-b661ddcc7250 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.427253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38969-8582-4c06-9a35-b661ddcc7250 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.427669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be38969-8582-4c06-9a35-b661ddcc7250 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.429596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 70/146] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:09.436074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.437156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.437156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.470943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:09.476622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:09.533351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd98576-608b-4a87-8a59-2beb51f118b8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:09.583144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468685f-c658-42bc-a314-63370c903591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:09.583917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468685f-c658-42bc-a314-63370c903591 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.584203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468685f-c658-42bc-a314-63370c903591 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.584477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468685f-c658-42bc-a314-63370c903591 None admin] PATH_INFO: `/v3/users/54b881f9b5fc4362a3e31f2060bf8350` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.588433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468685f-c658-42bc-a314-63370c903591 None admin] RBAC: Authorizing `identity:get_user(user_id=54b881f9b5fc4362a3e31f2060bf8350)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:09.590011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468685f-c658-42bc-a314-63370c903591 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:09.591774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 77/147] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:09 2022] GET /identity/v3/users/54b881f9b5fc4362a3e31f2060bf8350 => generated 289 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:09.597543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468685f-c658-42bc-a314-63370c903591 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:09.631957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:09.634650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 71/148] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:09.738542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468685f-c658-42bc-a314-63370c903591 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:09.776471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:09.777121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.777466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.777743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.778890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:09.780389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:09.782017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 78/149] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:09 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:09.787987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:09.904058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:09.910892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6c9080-3231-4323-9280-77d5180ebfe7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:09.956078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4028c5-5dd4-46ce-a4dd-21ad49e11a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:09.956845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4028c5-5dd4-46ce-a4dd-21ad49e11a45 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:09.957143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4028c5-5dd4-46ce-a4dd-21ad49e11a45 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:09.957412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4028c5-5dd4-46ce-a4dd-21ad49e11a45 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/54b881f9b5fc4362a3e31f2060bf8350/roles/1c08be9ad98543729654cd64c1c27930` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:09.963395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4028c5-5dd4-46ce-a4dd-21ad49e11a45 None admin] RBAC: Authorizing `identity:create_grant(project_id=b378b246cf9b49a9b0410534f7cc6d92, user_id=54b881f9b5fc4362a3e31f2060bf8350, role_id=1c08be9ad98543729654cd64c1c27930)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:09.964949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4028c5-5dd4-46ce-a4dd-21ad49e11a45 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:09.989188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 72/150] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:09 2022] PUT /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/users/54b881f9b5fc4362a3e31f2060bf8350/roles/1c08be9ad98543729654cd64c1c27930 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:10.305153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:10.446674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba7a5e-21b0-4ac8-afba-eae6ec0f99bc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.447174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba7a5e-21b0-4ac8-afba-eae6ec0f99bc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.447249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba7a5e-21b0-4ac8-afba-eae6ec0f99bc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.449983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 73/151] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:10.457861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47f441-dbdb-4d3f-9638-d0cd1762b182 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.458184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47f441-dbdb-4d3f-9638-d0cd1762b182 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.458413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca47f441-dbdb-4d3f-9638-d0cd1762b182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.461821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:10.475969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68dc994-2e18-4633-883c-98ca6abfed30 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:10.490460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ca47f441-dbdb-4d3f-9638-d0cd1762b182 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:10.492367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ca47f441-dbdb-4d3f-9638-d0cd1762b182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:10.557068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:10.558016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.558298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.558641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.592650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:10.598662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:10.599632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:10.600404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 79/152] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:10.611301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:10.633035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 74/153] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:10.734019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88719e3-069d-4ec3-9f8c-849600911d51 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.734357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88719e3-069d-4ec3-9f8c-849600911d51 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.734574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88719e3-069d-4ec3-9f8c-849600911d51 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.736031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 75/154] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:10.768612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d771cdb-b8c3-4043-afa2-2657f410342b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.769067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d771cdb-b8c3-4043-afa2-2657f410342b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.769343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d771cdb-b8c3-4043-afa2-2657f410342b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.807617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0d771cdb-b8c3-4043-afa2-2657f410342b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:10.809318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0d771cdb-b8c3-4043-afa2-2657f410342b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:10.848476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfebc97a-3dd0-47e0-952b-9fb5f2935109 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:10.916262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:10.917292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.917737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.918206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.919142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:10.919649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:10.925375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725b58f-78a9-4134-85b9-c046e3601776 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:10.945879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 80/155] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:10.949400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb906a3b-ddce-4f2c-9b85-e6e73127b72d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.949797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb906a3b-ddce-4f2c-9b85-e6e73127b72d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.950173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb906a3b-ddce-4f2c-9b85-e6e73127b72d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.954150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 81/156] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:10.958177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb906a3b-ddce-4f2c-9b85-e6e73127b72d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:10.969449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 76/157] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:10.977556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:10.977990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:10.978410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:10.999936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:11.011660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:11.012898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:11.033347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:11.192860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb906a3b-ddce-4f2c-9b85-e6e73127b72d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:11.202863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 77/158] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:11.252690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:11.253340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:11.253620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:11.253894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:11.255150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:11.256644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:11.258702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 82/159] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:10 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:11.264904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:11.311727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:11.434880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:11.442587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a9a58-7309-4f91-852f-950857d1e237 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:11.464081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41df3eda-c482-47ac-851c-27885492fe6d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:11.512351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:11.513009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:11.513305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:11.513562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:11.515367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:11.516016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:11.516294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:11.516567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:11.529729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:11.535247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:11.536627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:11.538763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:11.540626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 83/160] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:11.548322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:11.553477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:11.674319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:11.676531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:11.703246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbc63-35fb-4384-9fde-2b4c3ba01af5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:11.752386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:11.753011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:11.753292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:11.753565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:11.754239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:11.754546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:11.755989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:11.772606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 84/161] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:11.781901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:11.786440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:11.862344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 78/162] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 598 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:11.863501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:11.927791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:12.021337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db30c36-17a5-4d33-9de0-e1885441f9ff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.080369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99b9-8eb7-490f-89bd-8f505db84336 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.109659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.110328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.110613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.110891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.128725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:12.134247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:12.135241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:12.136116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 85/163] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:12.141510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:12.163900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.164838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.165350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.165827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] PATH_INFO: `/v3/users/91fe14f693174ef291042aa145475ad4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.167188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] RBAC: Authorizing `identity:get_user(user_id=91fe14f693174ef291042aa145475ad4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:12.177709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:12.187930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 79/164] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:11 2022] GET /identity/v3/users/91fe14f693174ef291042aa145475ad4 => generated 247 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:12.189074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:12.247914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:12.276073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14680875-9e2a-4776-bd3a-09879b020fcd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.322002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.322829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.322976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.323219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.339570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:12.343344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:12.344312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:12.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 86/165] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:12.345889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:12.478498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab06c69-3f05-4f0c-9c66-90c84bfffd21 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.507413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:12.524086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.524862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.525162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.525440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.526202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:12.526439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:12.532454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:12.543052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 87/166] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:12.544197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:12.546306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742b27b-edfe-4e41-985b-db3b64e0b7af None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.592635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:12.659521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.660125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757b5cbf-56ef-4630-bfcc-97d429c7fe58 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.660816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.661306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.661786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.663131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:12.664850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:12.666737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 80/167] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:12 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:12.667762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:12.703709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.704382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.704661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.704933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.775983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:12.784163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:12.785947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:12.925686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caccb7d-e98e-44ab-8320-e106342be73a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:12.983939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:12.984787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:12.985195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:12.985609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:12.986774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:12.987259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:12.989542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:12.995117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 88/168] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:12.996805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.003681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 81/169] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:13.009317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.127993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:13.222155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:13.238146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263349e-e13e-4909-9192-a8859b9b9479 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:13.250394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1a5698-c952-41b9-a052-2e529413eb8e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:13.310638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:13.311499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.311912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.312289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.313338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=597cf93046794f75bf6b7a42a09fb48c) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:13.313776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:13.317788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:13.335589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 82/170] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:13 2022] GET /identity/v3/role_assignments?user.id=597cf93046794f75bf6b7a42a09fb48c&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 608 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:13.338500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.346424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:13.347380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.347846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.348257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.351002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:13.353332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:13.371307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 89/171] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:12 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:13.372802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.629921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930234e1-1544-49a2-978d-0b741c0333b3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:13.647350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc05f8-49ef-4c12-b92b-79b73feafb71 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:13.648945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:13.695420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:13.696092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.696372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.696643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.698019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:13.700049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:13.701696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 83/172] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:13 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:13.707307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.714480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff5c31f-db94-4821-9892-4310add69e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:13.715336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5c31f-db94-4821-9892-4310add69e1b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.715639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5c31f-db94-4821-9892-4310add69e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.715916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff5c31f-db94-4821-9892-4310add69e1b None admin] PATH_INFO: `/v3/users/597cf93046794f75bf6b7a42a09fb48c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.721903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5c31f-db94-4821-9892-4310add69e1b None admin] RBAC: Authorizing `identity:get_user(user_id=597cf93046794f75bf6b7a42a09fb48c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:13.723468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff5c31f-db94-4821-9892-4310add69e1b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:13.725394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 90/173] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:13 2022] GET /identity/v3/users/597cf93046794f75bf6b7a42a09fb48c => generated 275 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:13.729689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d8fea-5891-4ab9-81da-db8257db465a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.730942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d8fea-5891-4ab9-81da-db8257db465a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.731220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d8fea-5891-4ab9-81da-db8257db465a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.732560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 91/174] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:13.734854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d8fea-5891-4ab9-81da-db8257db465a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.767381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:13.847398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318af19b-ce19-4246-b9bf-009c173649b0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:13.868250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d8fea-5891-4ab9-81da-db8257db465a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:13.893122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48a3e5-7826-4963-a6a5-06112ef49928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:13.894113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48a3e5-7826-4963-a6a5-06112ef49928 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.894469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48a3e5-7826-4963-a6a5-06112ef49928 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.895386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48a3e5-7826-4963-a6a5-06112ef49928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.913719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:13.914378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.914664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.914942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.915880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=91fe14f693174ef291042aa145475ad4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:13.916200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:13.918664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15605916-460b-437b-9844-dca4b8a9e1d6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:13.925701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 92/175] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:13 2022] GET /identity/v3/role_assignments?user.id=91fe14f693174ef291042aa145475ad4&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:13.927216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5734a-82d8-498e-8f70-6355991ef7bb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:13.927507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5734a-82d8-498e-8f70-6355991ef7bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:13.927822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5734a-82d8-498e-8f70-6355991ef7bb None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:13.933352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 93/176] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:13.933352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5734a-82d8-498e-8f70-6355991ef7bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:13.966518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.000750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48a3e5-7826-4963-a6a5-06112ef49928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:14.002390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48a3e5-7826-4963-a6a5-06112ef49928 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:14.051030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.071794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5734a-82d8-498e-8f70-6355991ef7bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.114358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc99df-54b4-4306-8a44-502ed4a47c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:14.114981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc99df-54b4-4306-8a44-502ed4a47c89 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.115271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc99df-54b4-4306-8a44-502ed4a47c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.115566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc99df-54b4-4306-8a44-502ed4a47c89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.125436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 84/177] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:14.126782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34027eef-d83b-427e-a3be-26de6dd1d672 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.127447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34027eef-d83b-427e-a3be-26de6dd1d672 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.127775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34027eef-d83b-427e-a3be-26de6dd1d672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.148126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-34027eef-d83b-427e-a3be-26de6dd1d672 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:14.149285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-34027eef-d83b-427e-a3be-26de6dd1d672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:14.197128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc99df-54b4-4306-8a44-502ed4a47c89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:14.198671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cc99df-54b4-4306-8a44-502ed4a47c89 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:14.228155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 85/178] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:14.229246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116d61-6f24-4096-9386-d559ba36f5fc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.229532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116d61-6f24-4096-9386-d559ba36f5fc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.229805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a116d61-6f24-4096-9386-d559ba36f5fc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.231353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 86/179] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:14.232394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5ff38-02d9-4672-83d7-bd508b1157c1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.232748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5ff38-02d9-4672-83d7-bd508b1157c1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.233020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5ff38-02d9-4672-83d7-bd508b1157c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.247358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.254402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c3e5ff38-02d9-4672-83d7-bd508b1157c1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:14.256248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c3e5ff38-02d9-4672-83d7-bd508b1157c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:14.295495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.314888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 94/180] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:14.316178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20ed8-8a06-4db0-8298-5ff88d75a89b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.316467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20ed8-8a06-4db0-8298-5ff88d75a89b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.316767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a20ed8-8a06-4db0-8298-5ff88d75a89b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.322162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 95/181] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:14.322162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20ed8-8a06-4db0-8298-5ff88d75a89b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:14.335007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 87/182] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:14.336454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.336812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.337159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.365342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:14.366932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:14.456283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 88/183] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:14.457482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:14.488233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a20ed8-8a06-4db0-8298-5ff88d75a89b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.530500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.551057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8182af3-6057-4b1f-849f-86d55fdad3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:14.552154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8182af3-6057-4b1f-849f-86d55fdad3ba None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.552539 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8182af3-6057-4b1f-849f-86d55fdad3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.552917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8182af3-6057-4b1f-849f-86d55fdad3ba None admin] PATH_INFO: `/v3/projects/612572751f1544f9ae840f085e09ad0d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.554389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8182af3-6057-4b1f-849f-86d55fdad3ba None admin] RBAC: Authorizing `identity:get_project(project_id=612572751f1544f9ae840f085e09ad0d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:14.556459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8182af3-6057-4b1f-849f-86d55fdad3ba None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:14.559302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 96/184] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:14 2022] GET /identity/v3/projects/612572751f1544f9ae840f085e09ad0d => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:14.564816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.565097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.565367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.579710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98961a1-3277-4c8b-a171-9c6e9dd90d6f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.583681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:14.584874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:14.617704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:14.619061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.619352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.619654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.620795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:14.622763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:14.629255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 89/185] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:14 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:14.629255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:14.634910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.656846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 97/186] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:14.663794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:14.753866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e82f7-1275-4094-a3e1-5044adb6362c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.775433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa2ee91-c296-4a4d-8b83-4fb1b9df1cad None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.796557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:14.797527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.797931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.798248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.800371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.811934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:14.814655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:14.815272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.815580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.815856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:14.816588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=612572751f1544f9ae840f085e09ad0d, user.id=597cf93046794f75bf6b7a42a09fb48c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:14.816812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:14.817651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:14.818453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:14.819066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:14.819066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:14.822622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 90/187] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:14.823359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:14.831598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 98/188] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:14 2022] GET /identity/v3/role_assignments?user.id=597cf93046794f75bf6b7a42a09fb48c&scope.project.id=612572751f1544f9ae840f085e09ad0d&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:14.831598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:14.901637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:14.947112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a376702-29fe-4500-ae5b-51b5b3e589ba None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.977488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef1aae4-9812-4df7-9b65-e1e746f46a83 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:14.989560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:14.990514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:14.990799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:14.991072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.003665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:15.008683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:15.009557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:15.010360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 91/189] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:15.012160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.019368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.020034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.020320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.020594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.021332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] RBAC: Adding query filter params (role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=597cf93046794f75bf6b7a42a09fb48c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:15.021643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.023143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.029879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 99/190] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:14 2022] GET /identity/v3/role_assignments?user.id=597cf93046794f75bf6b7a42a09fb48c&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.030737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.095515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.153316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8f0f3-726d-4440-87e9-0b3bb23bfeb0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.158936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e463d-28ca-4708-99f2-45a33e83cc57 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.170860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.197549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.198192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.198467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.198804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.200949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.201582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.201867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.202151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.202837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:15.203146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.204597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.210459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:15.218901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 100/191] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.218901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:15.219159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:15.220225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 92/192] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:15.221010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.328457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d7027c-40a6-49d4-9b9e-5342864df08d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.333286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d257c67-c46d-421e-a558-50f41eae82c7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.342878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.363933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.364542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.364820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.365091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.365750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:15.366059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.367449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.367578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.368129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.368405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.368670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.369758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.371251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.373018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 101/193] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.373577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.375980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 93/194] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.376783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.420299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.478724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144aee7-221b-405f-8142-1b924bd195bc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.478916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4607eb-bcb3-4ffd-bb6c-2419dad1a214 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.512157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.512790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.512915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.513066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.513334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] PATH_INFO: `/v3/users/60deb08fc80a45538a5a6cbeb22fd00c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.513422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.513679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.513954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.514614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:15.514923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.516323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.524323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 94/195] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.525026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.526587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] RBAC: Authorizing `identity:get_user(user_id=60deb08fc80a45538a5a6cbeb22fd00c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.528142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.533958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 102/196] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/users/60deb08fc80a45538a5a6cbeb22fd00c => generated 283 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.533958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.565136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.624424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.626452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf321ac-643f-4461-b35c-d066e9376fa4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.627917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baef738-8e0e-4c8c-80e3-185d1866cffc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.660356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.660965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.661179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.661285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.661504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.661804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.662082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.662171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.662353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] PATH_INFO: `/v3/users/60deb08fc80a45538a5a6cbeb22fd00c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.663463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] RBAC: Authorizing `identity:get_user(user_id=60deb08fc80a45538a5a6cbeb22fd00c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.663647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.664928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.666486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 103/197] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/users/60deb08fc80a45538a5a6cbeb22fd00c => generated 283 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.666486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.685969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 95/198] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:15 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:15.686780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.795508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b3c30-b034-4a9c-ace1-3a0a28b54706 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.808306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.818221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08dc92-5070-4226-aba2-48231edecfa5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:15.838401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.839330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.839708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.839998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] PATH_INFO: `/v3/users/54b881f9b5fc4362a3e31f2060bf8350` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.841243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] RBAC: Authorizing `identity:get_user(user_id=54b881f9b5fc4362a3e31f2060bf8350)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.842745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.844624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 104/199] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/users/54b881f9b5fc4362a3e31f2060bf8350 => generated 289 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:15.845208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:15.862599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:15.863172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:15.863453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:15.863758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:15.889927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:15.947622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:15.949858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:15.970032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c73b2-9d54-4765-98c6-17f6f357d290 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.011884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.012548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.012834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.013104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.076699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:16.089496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 96/200] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:16.090983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.104302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.107023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.145282 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:16.207208 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617d9e24-5f15-4faa-969c-9d8b731374c7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.228678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 105/201] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:16.230372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.244834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.245520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.245717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.246521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.316821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:16.325826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.327421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.350870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa559f5-b58b-4044-a93c-d64b2d4c5b29 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.393307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.394432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.395042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.395457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.396786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.398930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.401636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 106/202] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:16.404395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.407160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:16.463722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 97/203] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:16.464543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.545172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893afb68-32e1-475b-bfbb-6358835daa24 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.567401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:16.576075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7ba5a-c80b-4230-8550-18b124e6e49e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.593924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.594822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.595110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.595386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.596938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.598395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.601801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 107/204] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:16.602668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.617197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.617811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.618089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.618361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.619093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=60deb08fc80a45538a5a6cbeb22fd00c) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:16.619400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.620931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.626901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 98/205] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/role_assignments?user.id=60deb08fc80a45538a5a6cbeb22fd00c&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:16.627813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.740079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:16.804929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89120b66-6388-404e-8113-911f3ff2a088 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.831004 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3d6ec-1ec9-4d4b-848d-d234e27145c2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:16.855757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.856473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.856755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.857088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.858496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.860059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.861979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 108/206] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:16.877293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:16.892380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:16.893012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:16.893293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:16.893612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:16.894352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] RBAC: Adding query filter params (role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=60deb08fc80a45538a5a6cbeb22fd00c) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:16.894660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:16.896145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8128cbb8-41d8-4e6e-ba14-0609bad9bde1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:16.902709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 99/207] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/role_assignments?user.id=60deb08fc80a45538a5a6cbeb22fd00c&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:16.994892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4239f-4f1b-4683-a7da-f557bb90cd18 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:17.033286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:17.033904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:17.034191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:17.034464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:17.035209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] RBAC: Adding query filter params (role.id=1c08be9ad98543729654cd64c1c27930, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92, user.id=54b881f9b5fc4362a3e31f2060bf8350) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:17.035522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:17.037020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b160346-f553-4565-8fb7-618dfe8d60a7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:17.043703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 109/208] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:16 2022] GET /identity/v3/role_assignments?user.id=54b881f9b5fc4362a3e31f2060bf8350&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=1c08be9ad98543729654cd64c1c27930 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:18.066305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d8dd8-de74-4f36-957b-45a9feb98406 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.067331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d8dd8-de74-4f36-957b-45a9feb98406 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.067870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9d8dd8-de74-4f36-957b-45a9feb98406 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.069429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 100/209] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:18.075401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.076247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.076715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.083477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:18.097556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:18.099005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:18.100065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead383-153b-45de-a269-8bdeed2bfc7e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.100350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead383-153b-45de-a269-8bdeed2bfc7e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.100624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ead383-153b-45de-a269-8bdeed2bfc7e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.102087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 101/210] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:18.108530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.108816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.109088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.126999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:18.128135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:18.156082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:18.188131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 110/211] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:18.213712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 102/212] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:18.463058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:18.495453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:18.581681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98969c1-1e8e-4e3b-9198-850f2d827027 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:18.607617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5bb20-af2a-4285-8720-33145386aac0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:18.620212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:18.627717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:18.628563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.628984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.629808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.643392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:18.644558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:18.645292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.645667 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.646031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.647222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:18.648782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:18.649467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:18.650343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:18.651137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:18.651782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 103/213] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:18.658178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 111/214] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:18.659495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:18.660605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:18.685750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:18.778538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f8aa7-565f-43d6-a03d-86ec05604a03 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:18.783378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748fe67e-2f53-4db8-8b4a-be4ad734f144 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:18.816398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:18.817327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.817781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.818227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.819043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:18.819910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:18.820590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:18.820937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:18.821264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:18.821364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:18.822042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:18.822404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:18.824414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:18.833926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 112/215] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:18.839646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:18.852182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 104/216] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:18 2022] POST /identity/v3/users => generated 297 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:42:18.911198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:18.963035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2104bb-ac97-455e-a270-bb5acd1a867d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:19.003498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3932e19-d686-49cd-8f0d-b0c005b24581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:19.004555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3932e19-d686-49cd-8f0d-b0c005b24581 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.005020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3932e19-d686-49cd-8f0d-b0c005b24581 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.005471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3932e19-d686-49cd-8f0d-b0c005b24581 None admin] PATH_INFO: `/v3/users/99b635dd75a4467d8f77866aed7f6203` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.022974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3932e19-d686-49cd-8f0d-b0c005b24581 None admin] RBAC: Authorizing `identity:get_user(user_id=99b635dd75a4467d8f77866aed7f6203)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:19.024888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3932e19-d686-49cd-8f0d-b0c005b24581 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:19.027246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 113/217] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:18 2022] GET /identity/v3/users/99b635dd75a4467d8f77866aed7f6203 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:19.033782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:19.114426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4aceca-0169-4270-b144-20699bbc3ea5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.114946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4aceca-0169-4270-b144-20699bbc3ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.114946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4aceca-0169-4270-b144-20699bbc3ea5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.116435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 114/218] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:19.118480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:19.123521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a235a3-cbae-4574-a471-a1d071462304 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.123809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a235a3-cbae-4574-a471-a1d071462304 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.124085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a235a3-cbae-4574-a471-a1d071462304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.148545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-78a235a3-cbae-4574-a471-a1d071462304 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:19.150113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-78a235a3-cbae-4574-a471-a1d071462304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:19.153861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21580124-1978-43b2-b6d6-dacce44e2eb7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:19.212261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:19.216186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.216186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.216186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.253361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 115/219] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:19.300536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:19.302269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:19.303791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caf20e-1b95-47bf-a453-3b9f7b70098b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.304178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caf20e-1b95-47bf-a453-3b9f7b70098b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.304840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5caf20e-1b95-47bf-a453-3b9f7b70098b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.307264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 116/220] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:19.313891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.314289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.314661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.339203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:19.340979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:19.390383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:19.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:19.419638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 105/221] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:19.440096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:19.440880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 117/222] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:19.553279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:19.572857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e464a57-82ff-44d9-9917-a5a61fcda773 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:19.608920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:19.609963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.610435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.610919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.612404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:19.614044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:19.616309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 106/223] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:19 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:19.622088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:19.707981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:19.782669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2e9ea-b98f-482e-ae86-f434ae3e6fe2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:19.782862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad93684b-009c-40e2-a47c-e82755f032b2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:19.818661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:19.819230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.819516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.819822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.830505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:19.831109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:19.831386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:19.831958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:19.832311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:19.832695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=99b635dd75a4467d8f77866aed7f6203) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:19.833001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:19.834439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:19.837374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:19.838244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:19.839046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 118/224] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:19.840347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:19.848021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 107/225] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:19 2022] GET /identity/v3/role_assignments?user.id=99b635dd75a4467d8f77866aed7f6203&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:19.850385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:19.891699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.025348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136fe9b6-a3d4-46fb-8a22-9220f04ba574 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.067916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.069178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.069403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.069841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.077903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42c6079-0bbc-429d-87e0-95c858158bbd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.089298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.095355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:20.101307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:20.102278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:20.103074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 119/226] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:20.109375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.128887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.129516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.129779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.130054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.130723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:20.131029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.133132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.142535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 108/227] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:20.152565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.226339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.237615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561844d5-5aa5-481d-9f5e-685f519e45e1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.274960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.275674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.275961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.276235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.276914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:20.277743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.279137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.288104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 120/228] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:20.293632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.341953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13ffc17-fd86-4810-95d3-1ec1ce0c7037 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.403377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c0b68-500f-475f-b0cf-3270c973f782 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.428867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.429520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.429797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.430124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] PATH_INFO: `/v3/users/60deb08fc80a45538a5a6cbeb22fd00c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.432121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] RBAC: Authorizing `identity:get_user(user_id=60deb08fc80a45538a5a6cbeb22fd00c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.434158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.440066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 109/229] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:20 2022] GET /identity/v3/users/60deb08fc80a45538a5a6cbeb22fd00c => generated 283 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:20.440925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.441574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.441846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.442124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] PATH_INFO: `/v3/users/60deb08fc80a45538a5a6cbeb22fd00c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.447758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] RBAC: Authorizing `identity:get_user(user_id=60deb08fc80a45538a5a6cbeb22fd00c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.448028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.449428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.451238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 121/230] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:20 2022] GET /identity/v3/users/60deb08fc80a45538a5a6cbeb22fd00c => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:20.456992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.483924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.551566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.569592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365cbb98-2a63-4756-8d0a-50aa63316b45 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.619389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.620033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.620316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.621429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.622744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.624232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.628574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 122/231] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:20 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:20.634883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.672573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5dc673-ca76-49ef-ab56-c91aacab3378 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.736792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.737438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.737722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.738000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.744215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.745707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.747566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 110/232] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:20 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:20.755391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:20.758375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.770137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c434b-f1b2-4bd4-abc9-fd2a6fd98eb0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.818128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f2aa1d-63db-4be2-9269-7af8c1424d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.818808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2aa1d-63db-4be2-9269-7af8c1424d1f None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.819149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2aa1d-63db-4be2-9269-7af8c1424d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.819434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2aa1d-63db-4be2-9269-7af8c1424d1f None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/60deb08fc80a45538a5a6cbeb22fd00c/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.821732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2aa1d-63db-4be2-9269-7af8c1424d1f None admin] RBAC: Authorizing `identity:create_grant(project_id=4d929fefaf4848f089f0e80abc945d88, user_id=60deb08fc80a45538a5a6cbeb22fd00c, role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.823206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f2aa1d-63db-4be2-9269-7af8c1424d1f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:20.839889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 123/233] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:20 2022] PUT /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/60deb08fc80a45538a5a6cbeb22fd00c/roles/0cc328004b4e4f44905d38cb222157a9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:20.945162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19f94e-4242-4116-8080-3ef002e964f6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:20.954433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:20.987925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a07f2-f895-4edc-81f8-f169b630bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:20.988663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a07f2-f895-4edc-81f8-f169b630bc8c None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:20.988951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a07f2-f895-4edc-81f8-f169b630bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:20.989225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a07f2-f895-4edc-81f8-f169b630bc8c None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/60deb08fc80a45538a5a6cbeb22fd00c/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:20.991180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a07f2-f895-4edc-81f8-f169b630bc8c None admin] RBAC: Authorizing `identity:create_grant(project_id=4d929fefaf4848f089f0e80abc945d88, user_id=60deb08fc80a45538a5a6cbeb22fd00c, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:20.992681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a07f2-f895-4edc-81f8-f169b630bc8c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:21.018121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 111/234] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:20 2022] PUT /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/60deb08fc80a45538a5a6cbeb22fd00c/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:21.463948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e6e4-e23f-4c66-8ea5-c6b9698d5d89 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:21.465008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e6e4-e23f-4c66-8ea5-c6b9698d5d89 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:21.465499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812e6e4-e23f-4c66-8ea5-c6b9698d5d89 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:21.467063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 124/235] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:21.472478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa6c7e-80c4-4bda-ba45-835fcfffbeda None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:21.473313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa6c7e-80c4-4bda-ba45-835fcfffbeda None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:21.473792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa6c7e-80c4-4bda-ba45-835fcfffbeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:21.498056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9baa6c7e-80c4-4bda-ba45-835fcfffbeda None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:21.499638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9baa6c7e-80c4-4bda-ba45-835fcfffbeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:21.586331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 112/236] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:21.635834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480022b1-4c50-457e-9b1b-29d4157fefbd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:21.636543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480022b1-4c50-457e-9b1b-29d4157fefbd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:21.637022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480022b1-4c50-457e-9b1b-29d4157fefbd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:21.638588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 125/237] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:21.644022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:21.644548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:21.645045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:21.668548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:21.670079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:21.747252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 113/238] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:21.890766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480022b1-4c50-457e-9b1b-29d4157fefbd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:22.013508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480022b1-4c50-457e-9b1b-29d4157fefbd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:22.060108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:22.061110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.061595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.062094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.063401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:22.065164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:22.066946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 126/239] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:22.072603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:22.085402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:22.123241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:22.132137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:22.283817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2037f-0536-44dd-9407-b30662595fb4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:22.301647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eac774-50de-4baf-9d3a-00c890228150 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:22.362727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:22.363354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.374112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.374449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.375135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:22.376631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:22.381618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:22.382262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.382539 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.382809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.405093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:22.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:22.412621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:22.413622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:22.415699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 127/240] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:22.427599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 114/241] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:22 2022] POST /identity/v3/users => generated 293 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:22.428634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:22.545316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:22.590241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1764bfd-0f77-4d10-9a5a-97aa8ee30b2c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:22.628092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:22.628710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.628989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.629262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.629935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:22.630236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:22.631660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479aef80-4ee7-4614-af00-6146743e128e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:22.640958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 128/242] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:22.646514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:22.657959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2880-94ef-4bfb-a8e8-067080ec522c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.658233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2880-94ef-4bfb-a8e8-067080ec522c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.658508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a2880-94ef-4bfb-a8e8-067080ec522c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.661221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 129/243] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:22.666250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.666530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.666802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.687214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:22.688430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:22.758284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:22.777908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 130/244] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:22.786561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681951ed-6ae3-4c22-acdc-4f2ebddda0dd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:22.822760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:22.823497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:22.824423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:22.824793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] PATH_INFO: `/v3/users/f56195239e9d40218da2d54a7b162a42` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:22.840478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] RBAC: Authorizing `identity:get_user(user_id=f56195239e9d40218da2d54a7b162a42)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:22.842053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:22.844094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 115/245] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:22 2022] GET /identity/v3/users/f56195239e9d40218da2d54a7b162a42 => generated 297 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:22.849487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:22.935608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:22.972149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff81afc-332f-4714-996b-015a7192a295 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.011198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.012750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.013268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.013757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.067457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:23.086110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:23.088036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:23.167307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:23.206512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 131/246] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:23.213838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:23.217793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650d54c-a59a-470e-bf6a-8fe800c6bda8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.260467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.260467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.260858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.261431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.270496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:23.274075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:23.281588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:23.282549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:23.283365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:23.285974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 116/247] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:23.292306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:23.338520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bcd5f-5083-4860-ab42-fa16b6002860 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.379975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.380513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.380790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.381059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.382208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:23.383696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:23.385623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 132/248] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:23 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:23.392475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:23.417192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28e049-6490-4fd5-944b-9acc4537f87d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.440495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:23.454900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.455701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.456073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.456438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.457200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:23.457613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:23.459158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:23.469047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 117/249] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:23.476542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:23.541778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5203403-e66d-45b6-989e-719b1e79b312 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.603050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:23.627798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.628487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.628726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.629005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.629742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] RBAC: Adding query filter params (role.id=1c08be9ad98543729654cd64c1c27930, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=f56195239e9d40218da2d54a7b162a42) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:23.630050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:23.635647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:23.647426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 133/250] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:23 2022] GET /identity/v3/role_assignments?user.id=f56195239e9d40218da2d54a7b162a42&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=1c08be9ad98543729654cd64c1c27930 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:23.683002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4addda5f-d0f4-4d41-bd4a-982064968b7e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.724893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074bf15-c8e8-48c0-8447-ca08d1541c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.725555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074bf15-c8e8-48c0-8447-ca08d1541c81 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.725836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074bf15-c8e8-48c0-8447-ca08d1541c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.726116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074bf15-c8e8-48c0-8447-ca08d1541c81 None admin] PATH_INFO: `/v3/users/99b635dd75a4467d8f77866aed7f6203` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.727291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074bf15-c8e8-48c0-8447-ca08d1541c81 None admin] RBAC: Authorizing `identity:get_user(user_id=99b635dd75a4467d8f77866aed7f6203)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:23.728904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074bf15-c8e8-48c0-8447-ca08d1541c81 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:23.731487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 118/251] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:23 2022] GET /identity/v3/users/99b635dd75a4467d8f77866aed7f6203 => generated 297 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:23.737130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:23.816141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae4590-0a98-456d-94b2-b829956748ed None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.816447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae4590-0a98-456d-94b2-b829956748ed None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.816687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ae4590-0a98-456d-94b2-b829956748ed None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.818161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 119/252] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:23.825323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.825625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.825876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.847233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:23.848509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:23.886597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b53fd8-8806-48ca-b250-95b50f581520 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:23.889155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:23.957955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baaadd3-39aa-4aa0-baed-a097a71136af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:23.959678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 120/253] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:23.959993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baaadd3-39aa-4aa0-baed-a097a71136af None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:23.960276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baaadd3-39aa-4aa0-baed-a097a71136af None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:23.960548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baaadd3-39aa-4aa0-baed-a097a71136af None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:23.961737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baaadd3-39aa-4aa0-baed-a097a71136af None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:23.963279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baaadd3-39aa-4aa0-baed-a097a71136af None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:23.972048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 134/254] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:23 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:23.977615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:24.053206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:24.077137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99135f86-b587-4ac7-9e2d-e7d91a2349d0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:24.077432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99135f86-b587-4ac7-9e2d-e7d91a2349d0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:24.077683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99135f86-b587-4ac7-9e2d-e7d91a2349d0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:24.079042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 135/255] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:24.084164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:24.084446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:24.084770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:24.112467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:24.114684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:24.141370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9ee3d-29ad-4fa4-a533-cda73f2a1561 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:24.216794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 136/256] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:24.256253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:24.256937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:24.257217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:24.257492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/99b635dd75a4467d8f77866aed7f6203/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:24.259468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d929fefaf4848f089f0e80abc945d88, user_id=99b635dd75a4467d8f77866aed7f6203, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:24.260946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:24.288902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 121/257] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:23 2022] PUT /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/99b635dd75a4467d8f77866aed7f6203/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:24.331419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:24.465613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:24.469182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03656641-4a64-48c5-9fc5-6115b39de646 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:24.519427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:24.520110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:24.520388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:24.520658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:24.537224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:24.542061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:24.547472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:24.549245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:24.550588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 137/258] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:24.556150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:24.711014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bdc78e-d085-41c7-a8cc-230f3fc51d83 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:24.745158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8688bc4-36be-491c-89cf-25082bba5a97 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:24.753447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:24.783940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:24.784580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:24.784857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:24.785125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:24.808893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:24.814286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:24.815183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:24.816093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:24.820302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 122/259] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:24.821153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:24.845407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:24.846048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:24.846329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:24.846605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:24.847289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:24.847656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:24.849073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:24.874087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 138/260] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:24.888600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:24.966846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:25.017708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5674e75-d6c5-4a99-9d4e-e0ce33847b5c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:25.102629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:25.103245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:25.103521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:25.103818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:25.106367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:25.106672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:25.113569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:25.138336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 123/261] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:25.139850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5cefeb-70c0-4406-9e12-0e975f47f7ff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:25.147980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:25.217235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:25.254944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:25.255627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:25.255917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:25.256198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] PATH_INFO: `/v3/users/60deb08fc80a45538a5a6cbeb22fd00c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:25.257359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] RBAC: Authorizing `identity:get_user(user_id=60deb08fc80a45538a5a6cbeb22fd00c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:25.258830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:25.260628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 139/262] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:24 2022] GET /identity/v3/users/60deb08fc80a45538a5a6cbeb22fd00c => generated 283 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:25.273022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:25.317318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f6acd-dfc6-4204-b5c4-26ea48f4a9de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:25.376941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:25.377587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:25.377866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:25.378139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] PATH_INFO: `/v3/users/60deb08fc80a45538a5a6cbeb22fd00c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:25.380226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] RBAC: Authorizing `identity:get_user(user_id=60deb08fc80a45538a5a6cbeb22fd00c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:25.381688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:25.383446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 124/263] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:25 2022] GET /identity/v3/users/60deb08fc80a45538a5a6cbeb22fd00c => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:25.400599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:25.459913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:25.480909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a354c9a5-d0b9-4eca-9129-746a95f9e9ca None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:25.543218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:25.543798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:25.544086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:25.544367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:25.575471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f265a6-88e0-4437-a768-48212e0a282f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:25.587377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:25.648264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad88f1-f1f8-484e-8644-f1939ddf4e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:25.648891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad88f1-f1f8-484e-8644-f1939ddf4e4f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:25.649170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad88f1-f1f8-484e-8644-f1939ddf4e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:25.649438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad88f1-f1f8-484e-8644-f1939ddf4e4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:25.672241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:25.673748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:25.794217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad88f1-f1f8-484e-8644-f1939ddf4e4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:25.796419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad88f1-f1f8-484e-8644-f1939ddf4e4f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:25.824538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:25.842349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 140/264] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:25.843465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:26.017079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe32da-a3a4-413b-b638-bf68eddb207f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:26.075251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc60a9-17ae-42f0-ba37-4367adfc9516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:26.075960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc60a9-17ae-42f0-ba37-4367adfc9516 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.076234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc60a9-17ae-42f0-ba37-4367adfc9516 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.076512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc60a9-17ae-42f0-ba37-4367adfc9516 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.077661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc60a9-17ae-42f0-ba37-4367adfc9516 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:26.079116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc60a9-17ae-42f0-ba37-4367adfc9516 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:26.080921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 141/265] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:25 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:26.082142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a699672-b97d-46a6-8309-37747944c6c5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.082540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a699672-b97d-46a6-8309-37747944c6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.082809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a699672-b97d-46a6-8309-37747944c6c5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.084327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 142/266] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:26.092205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a699672-b97d-46a6-8309-37747944c6c5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:26.112924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 125/267] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:26.123207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.123510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.123786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.160389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:26.161626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:26.171637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:26.233444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:26.309532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 126/268] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:26.310383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:26.331611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a699672-b97d-46a6-8309-37747944c6c5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:26.412242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:26.412868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.413149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.413421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.414162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] RBAC: Adding query filter params (role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=60deb08fc80a45538a5a6cbeb22fd00c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:26.414471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:26.415954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:26.434607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 143/269] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:26 2022] GET /identity/v3/role_assignments?user.id=60deb08fc80a45538a5a6cbeb22fd00c&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:26.449588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824db40-9ed2-4a85-9b49-82212cbe60aa None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:26.500339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:26.501400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.501680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.501951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.508133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:26.509784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:26.512980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 127/270] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:26 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:26.519160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:26.668593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727d7bd-92f4-4de1-9e21-ead9c487b5b0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:26.697838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:26.711271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:26.720976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:26.721626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.721916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.722191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.722971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=60deb08fc80a45538a5a6cbeb22fd00c) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:26.723290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:26.724872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a7ab2-0a51-4ab3-a1cd-8cff0753198e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:26.727821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:26.732542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 144/271] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:26 2022] GET /identity/v3/role_assignments?user.id=60deb08fc80a45538a5a6cbeb22fd00c&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:26.733905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd0af8-239d-495a-b37b-a55a8f3c5499 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.734206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd0af8-239d-495a-b37b-a55a8f3c5499 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.734617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fd0af8-239d-495a-b37b-a55a8f3c5499 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.739571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 145/272] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:26.741810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78353b-06f4-4671-84d7-3735b0e0f81c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.742102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78353b-06f4-4671-84d7-3735b0e0f81c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78353b-06f4-4671-84d7-3735b0e0f81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.762252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-da78353b-06f4-4671-84d7-3735b0e0f81c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:26.763481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-da78353b-06f4-4671-84d7-3735b0e0f81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:26.855953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 146/273] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:26.949973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12bf92-8fa0-492f-ab9b-723a26667049 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.950160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12bf92-8fa0-492f-ab9b-723a26667049 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.950446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12bf92-8fa0-492f-ab9b-723a26667049 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.950895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4c3e9-5203-468b-91cf-30fafffbe191 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:26.952178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 147/274] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:26.957386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.957675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.957953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:26.989830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:26.991022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:26.993632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:26.994277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:26.994555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:26.994822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.008378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:27.014220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:27.015115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:27.016041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 128/275] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:27.021226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.079727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 148/276] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:27.108715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:27.142632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b940ffc5-14a3-486e-a32e-54996766b7e0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.188808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.189428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.189714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.189987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.190719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:27.191027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:27.192453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:27.204201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 129/277] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:27.206942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.224372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.264611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:27.323386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ce779-2908-4dcf-afc9-5817abd26155 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.343197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677da36-6222-49f3-8db5-e199f5fe9ad5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.360480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.361118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.361397 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.361667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] PATH_INFO: `/v3/users/5bb1fd99ef3d486db0604c70418cd803` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.375369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:27.376786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] RBAC: Authorizing `identity:get_user(user_id=5bb1fd99ef3d486db0604c70418cd803)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:27.378402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:27.380181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 149/278] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/users/5bb1fd99ef3d486db0604c70418cd803 => generated 293 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:27.382545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.383397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.383874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.384324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.386807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.387225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:27.388884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:27.390920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 130/279] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:27.397338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.494619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92de3f13-a34f-4f3a-bc1e-4e71a0c972b6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.496987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:27.503183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b6527-4131-4a89-b75d-5a2bc77c78fa None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.531205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.531847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.532129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.532402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.533135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, user.id=5bb1fd99ef3d486db0604c70418cd803, scope.system=all) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:27.533453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:27.535118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:27.538700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbea67-4b04-401d-aab0-a93cea35d983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.539306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbea67-4b04-401d-aab0-a93cea35d983 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.539618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbea67-4b04-401d-aab0-a93cea35d983 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.539894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbea67-4b04-401d-aab0-a93cea35d983 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.540106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 150/280] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/role_assignments?user.id=5bb1fd99ef3d486db0604c70418cd803&scope.system=all&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:27.540870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.544161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbea67-4b04-401d-aab0-a93cea35d983 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:27.545763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbea67-4b04-401d-aab0-a93cea35d983 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:27.573496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 131/281] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:27 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:42:27.576346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b57dd-7c30-4a8e-baa6-9715820dc292 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.576638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b57dd-7c30-4a8e-baa6-9715820dc292 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.576914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b57dd-7c30-4a8e-baa6-9715820dc292 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.578342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 132/282] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:27.585915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.586249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.586535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.607706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:27.609192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:27.670434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:27.695930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 133/283] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:27.698127 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b8a35-969e-4003-a3e5-834539378f9e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.739048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.739730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.740015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.740290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.754014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:27.760994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:27.762279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:27.764038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 151/284] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:27.767470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.877345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0cd6e-9c49-46fd-8724-947f85a7876b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:27.914308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:27.915267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:27.915785 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:27.916234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:27.917090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:27.917677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:27.919230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:27.928312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 134/285] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:27.934520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:27.966714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:27.976214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.035106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:28.057880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a6fc55-ff3e-41ea-b347-ce61c18ae2cd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.095681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945929a5-7909-4774-a437-aa553febe887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.096352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945929a5-7909-4774-a437-aa553febe887 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.097007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945929a5-7909-4774-a437-aa553febe887 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.097292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945929a5-7909-4774-a437-aa553febe887 None admin] PATH_INFO: `/v3/users/f56195239e9d40218da2d54a7b162a42` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.098473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945929a5-7909-4774-a437-aa553febe887 None admin] RBAC: Authorizing `identity:get_user(user_id=f56195239e9d40218da2d54a7b162a42)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:28.099955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945929a5-7909-4774-a437-aa553febe887 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:28.102662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 152/286] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/users/f56195239e9d40218da2d54a7b162a42 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:28.109260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945929a5-7909-4774-a437-aa553febe887 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.129899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65667b34-016b-4d7c-a763-e0450eb6bdee None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.202772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.202772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.202772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.202772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.211891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:28.224121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:28.225059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 135/287] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:28.225848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.267574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:28.270400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945929a5-7909-4774-a437-aa553febe887 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.331832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef721b-2dde-466e-89b5-dddebf3f7fda None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.335347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.336210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.336639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.337057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.341120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:28.342646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:28.344484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:28.346855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 153/288] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:28 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:28.352052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.370968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.371996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.372501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.372969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.373831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:28.374334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:28.376041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:28.390281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 136/289] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:28.391909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.511873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d726e-fb9e-4305-b5d7-def53f03f45f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.550155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.551143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.551690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.552177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] PATH_INFO: `/v3/users/99b635dd75a4467d8f77866aed7f6203` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.553535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] RBAC: Authorizing `identity:get_user(user_id=99b635dd75a4467d8f77866aed7f6203)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:28.555324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:28.557161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 137/290] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:28 2022] GET /identity/v3/users/99b635dd75a4467d8f77866aed7f6203 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:28.563488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.616098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:28.654016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef5a58-b875-489d-894f-965a61c9344f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.669449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497fad30-0548-43c5-8106-0d1b28f732f0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:28.704012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.704964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.705487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.705961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.777292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:28.779108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:28.795325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:28.796054 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:28.796332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:28.796606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/f56195239e9d40218da2d54a7b162a42/roles/1c08be9ad98543729654cd64c1c27930` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:28.808669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d929fefaf4848f089f0e80abc945d88, user_id=f56195239e9d40218da2d54a7b162a42, role_id=1c08be9ad98543729654cd64c1c27930)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:28.810205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:28.827140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:28.840470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 154/291] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:42:28 2022] PUT /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88/users/f56195239e9d40218da2d54a7b162a42/roles/1c08be9ad98543729654cd64c1c27930 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:42:28.913216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 138/292] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:42:28.919352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:28.953110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:29.040229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708d369-154a-4d44-b7ec-97c0e12354d6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:29.076299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d8b2f-a8f3-4256-8f54-1b94556a85ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:29.076939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d8b2f-a8f3-4256-8f54-1b94556a85ab None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:29.077220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d8b2f-a8f3-4256-8f54-1b94556a85ab None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:29.077497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d8b2f-a8f3-4256-8f54-1b94556a85ab None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:29.078676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d8b2f-a8f3-4256-8f54-1b94556a85ab None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:29.080173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d8b2f-a8f3-4256-8f54-1b94556a85ab None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:29.081903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 155/293] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:28 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:29.087928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:29.218446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13520e68-e8fe-433b-9a45-5aa347636548 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:29.263482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:29.264460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:29.264936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:29.265410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:29.266350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=99b635dd75a4467d8f77866aed7f6203) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:29.266854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:29.268514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56755fc-c8d3-43ad-9743-cdf6b6172608 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:29.277341 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:29.281891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 139/294] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:29 2022] GET /identity/v3/role_assignments?user.id=99b635dd75a4467d8f77866aed7f6203&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:29.985994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3066978-a33a-44bf-a220-cdfd1a24c9f8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:29.986832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3066978-a33a-44bf-a220-cdfd1a24c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:29.987249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3066978-a33a-44bf-a220-cdfd1a24c9f8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:29.988983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 156/295] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:29.994501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821dacc-cc5f-4c33-9745-d66ed76a896a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:29.995046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821dacc-cc5f-4c33-9745-d66ed76a896a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:29.996093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821dacc-cc5f-4c33-9745-d66ed76a896a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.015064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a821dacc-cc5f-4c33-9745-d66ed76a896a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:30.016576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a821dacc-cc5f-4c33-9745-d66ed76a896a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:30.092324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 140/296] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:30.280160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d28f5b-291e-4cac-af9d-114e03d594fc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.280796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d28f5b-291e-4cac-af9d-114e03d594fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.281262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d28f5b-291e-4cac-af9d-114e03d594fc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.282795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 157/297] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:30.288420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e5c6e-cb36-4a08-b71a-f54130cd01d4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.288959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e5c6e-cb36-4a08-b71a-f54130cd01d4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.289425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e5c6e-cb36-4a08-b71a-f54130cd01d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.308871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2e4e5c6e-cb36-4a08-b71a-f54130cd01d4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:30.310330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2e4e5c6e-cb36-4a08-b71a-f54130cd01d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:30.379360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d28f5b-291e-4cac-af9d-114e03d594fc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:30.390521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 141/298] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:30.479260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:30.494328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d28f5b-291e-4cac-af9d-114e03d594fc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:30.502911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73886ea0-e73d-4d90-9151-51f6b990ff9a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.503480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73886ea0-e73d-4d90-9151-51f6b990ff9a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.504009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73886ea0-e73d-4d90-9151-51f6b990ff9a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.505576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 142/299] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:30.510716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.511799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.512601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.531022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:30.531628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.531915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.532186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.532367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:30.534282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:30.545053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:30.549949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:30.550837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:30.552104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 158/300] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:30.556730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:30.609269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:30.625605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 143/301] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:30.672448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8820e-c67e-4d49-af57-ae4eb3d71c2b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:30.684626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:30.718329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:30.718996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.719279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.719569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.720491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:30.720794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:30.722865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:30.733725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 159/302] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:30.738750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:30.763904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:30.811826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f6ec3-7965-489f-9af3-06621eaa0d16 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:30.857537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:30.858598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.859330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.860060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.862319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75925a79-10aa-4add-9049-3d6688afcfbb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:30.874497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:30.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:30.882617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:30.883464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:30.887684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 144/303] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:30.891430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:30.911644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:30.912311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:30.912589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:30.912860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] PATH_INFO: `/v3/users/5bb1fd99ef3d486db0604c70418cd803` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:30.914016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] RBAC: Authorizing `identity:get_user(user_id=5bb1fd99ef3d486db0604c70418cd803)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:30.915454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:30.917053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 160/304] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:30 2022] GET /identity/v3/users/5bb1fd99ef3d486db0604c70418cd803 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:30.918485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:30.953785 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:31.010775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:31.011880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52204cc5-9d1c-4455-a0fe-7299a7b69bd7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.032215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9086814b-9dbb-443d-b1ed-455ed2ff37e1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.046333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.046939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.047218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.047475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.048163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:31.048463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:31.049838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:31.063015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 145/305] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:31.063814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:31.069055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.069920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.070507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.070960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.072386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:31.073872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:31.079579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 161/306] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:31.079579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:31.171956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ee8a42-abf7-46a4-a8f4-2406eb3b2fd7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.185017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16837d-5c55-4f43-8271-0544427ecff9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.185367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:31.207996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.208697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.209002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.209255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] PATH_INFO: `/v3/system/users/5bb1fd99ef3d486db0604c70418cd803/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.210658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5bb1fd99ef3d486db0604c70418cd803, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:31.212082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:31.220828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 146/307] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Sep 7 15:42:31 2022] PUT /identity/v3/system/users/5bb1fd99ef3d486db0604c70418cd803/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:31.223344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.224069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.224381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.224651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] PATH_INFO: `/v3/users/1c35604a3ace4517808a3b828f0ab510` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.230304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:31.248476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] RBAC: Authorizing `identity:get_user(user_id=1c35604a3ace4517808a3b828f0ab510)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:31.251352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:31.254097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 162/308] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:31 2022] GET /identity/v3/users/1c35604a3ace4517808a3b828f0ab510 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:31.270235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:31.288758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:31.367116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b0c85-8da2-4f19-b0c8-da319024a276 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.405449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea6ae6-65ca-4ff4-8703-c19262a63dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.406075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea6ae6-65ca-4ff4-8703-c19262a63dcf None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.406353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea6ae6-65ca-4ff4-8703-c19262a63dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.406630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea6ae6-65ca-4ff4-8703-c19262a63dcf None admin] PATH_INFO: `/v3/groups` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.407286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea6ae6-65ca-4ff4-8703-c19262a63dcf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:31.408759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea6ae6-65ca-4ff4-8703-c19262a63dcf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:31.412508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bebddc-1aec-4e4d-b7d5-eb592918aa66 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.427424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 147/309] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Sep 7 15:42:31 2022] POST /identity/v3/groups => generated 229 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:31.429241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a9f4f-0340-4113-ae0d-ee7488d4c965 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.429241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a9f4f-0340-4113-ae0d-ee7488d4c965 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.429241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a9f4f-0340-4113-ae0d-ee7488d4c965 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.430422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 148/310] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:31.436977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.437285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.437530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.451945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.452572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.452848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.453118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.453913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] RBAC: Adding query filter params (role.id=1c08be9ad98543729654cd64c1c27930, user.id=1c35604a3ace4517808a3b828f0ab510, scope.system=all) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:31.454224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:31.455725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:31.457640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:31.459144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:31.461279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 163/311] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 15:42:31 2022] GET /identity/v3/role_assignments?user.id=1c35604a3ace4517808a3b828f0ab510&scope.system=all&role.id=1c08be9ad98543729654cd64c1c27930 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:31.498705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:31.554762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 149/312] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:31.825088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:31.897806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:31.931306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58851c25-80a4-4c65-abf4-64d60be8c4e5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:31.967352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:31.968411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:31.968915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:31.969401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:31.982833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:31.988578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:31.989480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:31.990287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:31.992894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 164/313] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:32.004665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:32.112430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6249b6b-3000-462f-9e92-4cca33dd0fe9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:32.148259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:32.149248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:32.149733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:32.150246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:32.151105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:32.152171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:32.154287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:32.163617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 150/314] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:32.172248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:32.278793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f30b22-9498-465c-a894-7373cc6e81fc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:32.308988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:32.315291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:32.315973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:32.316257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:32.316536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] PATH_INFO: `/v3/users/f56195239e9d40218da2d54a7b162a42` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:32.317682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] RBAC: Authorizing `identity:get_user(user_id=f56195239e9d40218da2d54a7b162a42)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:32.319133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:32.320760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 165/315] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:32 2022] GET /identity/v3/users/f56195239e9d40218da2d54a7b162a42 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:32.326350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:32.358858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:32.441639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce10fcd-ba0f-4d55-a926-068583ba4d03 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:32.477021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:32.477689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:32.477972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:32.478301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:32.550009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:32.551477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:32.583697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:32.663308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 151/316] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:32.669194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:32.775471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fd429-e6d8-4c2f-92f4-2527ffa31694 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:32.810213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c844af-b48f-49aa-8578-8a3569a7a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:32.810857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c844af-b48f-49aa-8578-8a3569a7a9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:32.811137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c844af-b48f-49aa-8578-8a3569a7a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:32.811409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c844af-b48f-49aa-8578-8a3569a7a9f9 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:32.812552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c844af-b48f-49aa-8578-8a3569a7a9f9 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:32.814024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c844af-b48f-49aa-8578-8a3569a7a9f9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:32.815655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 166/317] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:32 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:32.821535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:32.924213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eeb756-fb14-4e18-b7d9-e8a6a1d7d2ab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:32.951818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:32.960694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:32.961308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:32.961587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:32.961855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:32.962580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] RBAC: Adding query filter params (role.id=1c08be9ad98543729654cd64c1c27930, scope.project.id=4d929fefaf4848f089f0e80abc945d88, user.id=f56195239e9d40218da2d54a7b162a42) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:32.962884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:32.964850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9140a6-06eb-400f-8154-afd09c8f6d0d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:32.970781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 152/318] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:42:32 2022] GET /identity/v3/role_assignments?user.id=f56195239e9d40218da2d54a7b162a42&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=1c08be9ad98543729654cd64c1c27930 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:33.529810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00efed-c74f-4dc1-bb8e-dfea997359c8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:33.530289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00efed-c74f-4dc1-bb8e-dfea997359c8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:33.530289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee00efed-c74f-4dc1-bb8e-dfea997359c8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:33.531775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 167/319] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:33.537321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66b0d1-05ff-40f6-acbe-e7e5b612e075 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:33.537762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66b0d1-05ff-40f6-acbe-e7e5b612e075 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:33.537938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66b0d1-05ff-40f6-acbe-e7e5b612e075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:33.556389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-db66b0d1-05ff-40f6-acbe-e7e5b612e075 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:33.557570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-db66b0d1-05ff-40f6-acbe-e7e5b612e075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:33.632512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 153/320] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:33.726386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdfdbc-84a5-4989-b370-589c5564fb4a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:33.726682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdfdbc-84a5-4989-b370-589c5564fb4a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:33.726948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdfdbc-84a5-4989-b370-589c5564fb4a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:33.728347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 168/321] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:33.729504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552f702-d50e-49a7-b6ce-ebffe6c9b71a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:33.730136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552f702-d50e-49a7-b6ce-ebffe6c9b71a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:33.730440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552f702-d50e-49a7-b6ce-ebffe6c9b71a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:33.731878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 154/322] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:33.733513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:33.733807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:33.734087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:33.736626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:33.736915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:33.737232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:33.752805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:33.754004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:33.755689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:33.756855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:33.823867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:33.828936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 155/323] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:33.835359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 169/324] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:33.901302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:33.954591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.002276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53d649-77b9-4605-8490-9f3ed256e3f9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.035618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.036319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.036598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.036867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.046664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:34.051469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:34.052351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:34.053168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:34.055850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 156/325] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:34.058357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.098766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.163475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c0f47-f04c-4e86-9879-8f26be5175d9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.197384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.197995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.198275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.199066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.199755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:34.200062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.201451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.202661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee433a8c-8b80-40bd-9419-9902b1e72b4f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.209320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 170/326] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:34.210106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.219211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.240825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.241442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.241715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.241983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.251682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:34.256461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:34.257335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:34.258127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 157/327] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:34.258798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.270960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.311784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399668d-1107-4dc8-ae6c-9a68c002b6ba None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.343850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.344478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.344758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.345034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.346131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.347588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.349196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 171/328] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:34.353438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.358112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6701cf-d6a6-407b-879f-4fc446daa013 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.391040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.391697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.391979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.392249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] PATH_INFO: `/v3/users/5bb1fd99ef3d486db0604c70418cd803` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.393362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] RBAC: Authorizing `identity:get_user(user_id=5bb1fd99ef3d486db0604c70418cd803)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.394823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.396421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 158/329] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/users/5bb1fd99ef3d486db0604c70418cd803 => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:34.397152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.439869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.454751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482bb7f9-37bc-4c93-a20e-9831139dd54e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.480314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.488110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.488714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.488989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.489257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.489906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] RBAC: Adding query filter params (name=reader) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:34.490207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.491590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.500075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 172/330] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:34.500862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80442056-68dd-4ebc-9e2c-e7dae38d01e3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.500967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.538911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.539510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.539823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.540093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] PATH_INFO: `/v3/groups` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.540703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.542153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.556531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 159/331] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Sep 7 15:42:34 2022] POST /identity/v3/groups => generated 226 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:34.567826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.611901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb69e447-60d0-4ff1-bcbc-ac338dbbc9de None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.657595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.658248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.658545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.658826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.659589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] RBAC: Adding query filter params (role.id=09d9b803d29e48b28dfc0fcb6d976e31, user.id=5bb1fd99ef3d486db0604c70418cd803, scope.system=all) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:34.659910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.661390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.674041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.674663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 173/332] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/role_assignments?user.id=5bb1fd99ef3d486db0604c70418cd803&scope.system=all&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:34.677707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827e70fd-72cb-4913-b9b1-5027bfb6526f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.721131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d309f9-c119-4147-8f67-370a2862efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.722039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d309f9-c119-4147-8f67-370a2862efcf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.722484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d309f9-c119-4147-8f67-370a2862efcf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.722896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d309f9-c119-4147-8f67-370a2862efcf None admin] PATH_INFO: `/v3/users/1c35604a3ace4517808a3b828f0ab510` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.724415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d309f9-c119-4147-8f67-370a2862efcf None admin] RBAC: Authorizing `identity:get_user(user_id=1c35604a3ace4517808a3b828f0ab510)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.726046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d309f9-c119-4147-8f67-370a2862efcf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.728355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 160/333] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:34 2022] GET /identity/v3/users/1c35604a3ace4517808a3b828f0ab510 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:34.734746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:34.793914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:34.859359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d96828-6e1f-4822-ac0f-c957e9e0873d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:34.897063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5c83a-a0c4-4a2f-9299-1865a89ced8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:34.898049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c83a-a0c4-4a2f-9299-1865a89ced8a None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:34.898541 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c83a-a0c4-4a2f-9299-1865a89ced8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:34.898820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5c83a-a0c4-4a2f-9299-1865a89ced8a None admin] PATH_INFO: `/v3/system/users/1c35604a3ace4517808a3b828f0ab510/roles/1c08be9ad98543729654cd64c1c27930` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:34.900286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5c83a-a0c4-4a2f-9299-1865a89ced8a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1c35604a3ace4517808a3b828f0ab510, role_id=1c08be9ad98543729654cd64c1c27930)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:34.901693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5c83a-a0c4-4a2f-9299-1865a89ced8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:34.912846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 174/334] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Sep 7 15:42:34 2022] PUT /identity/v3/system/users/1c35604a3ace4517808a3b828f0ab510/roles/1c08be9ad98543729654cd64c1c27930 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:37.089238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8151f4a-8952-4cde-a9cc-9474a381a7b0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.089779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8151f4a-8952-4cde-a9cc-9474a381a7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.090040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8151f4a-8952-4cde-a9cc-9474a381a7b0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.091515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 161/335] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:37.097244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf64fa-4694-4d74-b6f5-96cc72264c3a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.097683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf64fa-4694-4d74-b6f5-96cc72264c3a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.097886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decf64fa-4694-4d74-b6f5-96cc72264c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.120272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-decf64fa-4694-4d74-b6f5-96cc72264c3a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:37.121473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-decf64fa-4694-4d74-b6f5-96cc72264c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:37.202959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 175/336] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:37.492929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca075138-e2ab-4dca-9010-058f38704973 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.492929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca075138-e2ab-4dca-9010-058f38704973 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.492929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca075138-e2ab-4dca-9010-058f38704973 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.493700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 162/337] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:37.499966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d177e3-9420-47a0-81cd-1f1c8a8e356c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.500491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d177e3-9420-47a0-81cd-1f1c8a8e356c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.501005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d177e3-9420-47a0-81cd-1f1c8a8e356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.525068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c2d177e3-9420-47a0-81cd-1f1c8a8e356c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:37.526272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c2d177e3-9420-47a0-81cd-1f1c8a8e356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:37.580810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:37.625811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 176/338] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:37.747358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4188-7bda-47f2-a9bc-fb668ec258ae None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.747988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4188-7bda-47f2-a9bc-fb668ec258ae None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.748466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd4188-7bda-47f2-a9bc-fb668ec258ae None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.750288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 163/339] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:37.756135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862f97-f1aa-4332-b6c5-bf0699179618 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.756718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862f97-f1aa-4332-b6c5-bf0699179618 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.757208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c862f97-f1aa-4332-b6c5-bf0699179618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.779394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8c862f97-f1aa-4332-b6c5-bf0699179618 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:37.779744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4188-7bda-47f2-a9bc-fb668ec258ae None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:37.781241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8c862f97-f1aa-4332-b6c5-bf0699179618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:37.871385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 177/340] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:37.873174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c66124-7b6f-4267-b6e8-ecde5552d5e2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.874918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c66124-7b6f-4267-b6e8-ecde5552d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.876221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c66124-7b6f-4267-b6e8-ecde5552d5e2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.880358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 178/341] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:37.888895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:37.889178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:37.889450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:37.928345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:37.929524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:37.968272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd4188-7bda-47f2-a9bc-fb668ec258ae None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.030983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.031988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.032464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.032947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.054919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:38.056488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 179/342] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:38.061778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:38.062673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:38.063468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:38.065443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.075993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 164/343] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:38.101468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.140076 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:38.172598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:38.259277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb1b8c7-4e07-4d05-af83-c098aeb423d1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.327187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68a62d-8864-4069-adc9-43ffc58c997e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.336898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.337554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.337838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.338114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.356820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:38.361746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:38.362613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:38.363422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:38.368750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 180/344] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:38.371230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.412246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.412882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.413168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.413443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.414116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:38.414422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:38.415853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:38.424569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 165/345] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:38.425405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.506859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:38.519255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619a60a5-cc6c-4ca8-9e35-09dd3ff1f96f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.556484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:38.563805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.564697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.565358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.565644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.568179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda841c9-21d0-4595-9561-749edc345a7f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.579166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:38.585785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:38.586701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:38.587652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 181/346] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:38.588327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.615375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.616097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.616437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.616710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.617432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:38.617744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:38.619265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:38.630438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 166/347] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:38.631099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.715123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8e436c-5940-4097-acc2-331d405893d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.755054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d602ed9-49c2-4523-b5c1-71d030e19bd0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.759871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.760493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.761297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.761573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.791446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:38.803575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:38.804231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:38.804509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:38.804781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:38.821833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:38.829772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:38.831816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 07 15:42:38.833708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:38.835791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 167/348] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:38.837975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:38.847729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:38.849340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:38.857563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:38.962564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c99e62a-4736-419d-99b2-29321b88d61c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:38.970271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 182/349] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:38.971254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.006891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.007512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.007816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.008090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.008768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:39.009060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.010427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.019816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 168/350] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:39.020659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.049347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.099955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f38fd-657c-459e-8d7f-8a624c91eb56 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.103174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.138080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7e0bd-271f-4d7f-a18f-f3ad0462c7fd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.146297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723abe2e-4fe4-4440-9a57-8ccb9db90f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.146915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723abe2e-4fe4-4440-9a57-8ccb9db90f7e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.147685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723abe2e-4fe4-4440-9a57-8ccb9db90f7e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.147963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723abe2e-4fe4-4440-9a57-8ccb9db90f7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.189145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.191204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.191204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.191204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.191204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:39.191501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.193086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.207612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 169/351] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:39.207612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.252191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723abe2e-4fe4-4440-9a57-8ccb9db90f7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.254301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723abe2e-4fe4-4440-9a57-8ccb9db90f7e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.327189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.330623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97f639-6b2a-4702-8f5c-3fab59c9ebe3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.369363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.369923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.370259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.370531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.371696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.373146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.375176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 170/352] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:39.375982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.385276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.395065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 183/353] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:39.396570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2daca8-c077-47c2-96c9-f240a8a3b9ab None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.397201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2daca8-c077-47c2-96c9-f240a8a3b9ab None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.397701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2daca8-c077-47c2-96c9-f240a8a3b9ab None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.399476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 184/354] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:39.400388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2daca8-c077-47c2-96c9-f240a8a3b9ab None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.492616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ac4c9-01d5-4309-9e55-dfd5d3bfdf83 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.515492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2daca8-c077-47c2-96c9-f240a8a3b9ab None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.532393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e12f2a-11ca-4f9f-ad10-61f3eaccf237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.533408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e12f2a-11ca-4f9f-ad10-61f3eaccf237 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.533954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e12f2a-11ca-4f9f-ad10-61f3eaccf237 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.534319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e12f2a-11ca-4f9f-ad10-61f3eaccf237 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.553635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.554265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.554542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.554811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] PATH_INFO: `/v3/users/1c35604a3ace4517808a3b828f0ab510` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.556533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] RBAC: Authorizing `identity:get_user(user_id=1c35604a3ace4517808a3b828f0ab510)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.558089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.559926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 185/355] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/users/1c35604a3ace4517808a3b828f0ab510 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:39.560939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.587637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.623202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e12f2a-11ca-4f9f-ad10-61f3eaccf237 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.624948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e12f2a-11ca-4f9f-ad10-61f3eaccf237 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.628807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.685240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e265074b-73aa-4daf-87d4-34735282c6f6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.730513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.731157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.731433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.731759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.739425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.740946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.742554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 186/356] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:39.748441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.762335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 171/357] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:39.763478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034181-9956-4364-a1d0-f8b9d376eb38 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.763878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034181-9956-4364-a1d0-f8b9d376eb38 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.764263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9034181-9956-4364-a1d0-f8b9d376eb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.788649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f9034181-9956-4364-a1d0-f8b9d376eb38 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:39.789868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f9034181-9956-4364-a1d0-f8b9d376eb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:39.849552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:39.868604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 172/358] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:39.871145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebd5b9-1fd9-435a-80dc-90fbbf3f7dbe None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.871145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebd5b9-1fd9-435a-80dc-90fbbf3f7dbe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.871145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ebd5b9-1fd9-435a-80dc-90fbbf3f7dbe None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.872269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449c75b1-5a0a-4f1e-8bc8-5d5fa43fa7cf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:39.873384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 173/359] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:39.874033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebd5b9-1fd9-435a-80dc-90fbbf3f7dbe None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.920173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:39.920818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:39.921168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:39.921450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:39.922574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:39.924077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:39.925701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 187/360] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:39.926925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:39.930661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.023364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ebd5b9-1fd9-435a-80dc-90fbbf3f7dbe None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.059526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67828eaf-90fd-4ff0-91ce-4087822744e4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.068895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.069703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.070166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.070615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.071971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] RBAC: Adding query filter params (role.id=1c08be9ad98543729654cd64c1c27930, user.id=1c35604a3ace4517808a3b828f0ab510, scope.system=all) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:40.072428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.074931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.092596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 174/361] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/role_assignments?user.id=1c35604a3ace4517808a3b828f0ab510&scope.system=all&role.id=1c08be9ad98543729654cd64c1c27930 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.093464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.109205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.109987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.110862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.111152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.112676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:40.113286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.115224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1de11e-fe75-4377-8003-d61bed4598d2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.123061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 188/362] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:39 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.124437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.124668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.124947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.147733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:40.149029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:40.172876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.252576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 189/363] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:40.253473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.270357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.287658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35549147-d8fe-4a06-8093-d6efe5b16a54 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.364130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.364802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.365088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.365820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.367351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.369730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.372431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 175/364] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.373686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.391826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d96efec-3463-4786-af4d-3f8c3b841f9f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.444216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.445179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.445651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.446158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.449180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.451279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 190/365] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.452243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.485004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.517225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759058e-e199-499a-a82b-b6d54b158be9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.560738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.561688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.562889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.563177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.579286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:40.584743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:40.585646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:40.586487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:40.590409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 176/366] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:40.590983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.607205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c463cef6-f459-44a7-ac45-3190b4b2d6ac None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.625406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.655399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.656033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.656319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.656599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.657810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.659270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.661330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 191/367] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.662303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.728271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa994ed4-e4e2-4cd1-a83d-c72fa27fb729 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.758541 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.776413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.777280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.777559 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.777830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.778981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=4d929fefaf4848f089f0e80abc945d88) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:40.779288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.781280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.785791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b49e539-d9e3-42fc-a23e-b531029c137b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.788979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 177/368] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.790049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.826517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.827175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.827459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.827780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:40.828467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:40.828774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:40.830207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:40.841353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 192/369] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:40.842290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:40.917916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:40.940697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ed7b7-5b0f-4c9e-af16-951f96b89e43 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.979244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa4291-541f-4981-8133-1479e3adde13 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:40.989534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:40.990494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:40.990953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:40.991518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.008155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:41.014855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:41.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:41.016672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:41.019233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 178/370] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:41.020323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.029384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:41.030124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:41.030474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:41.030814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.031653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:41.032031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:41.033566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:41.040238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 193/371] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:40 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:41.041027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.054186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:41.167162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bef5b8-c66f-49bc-be9c-82c1c70a9191 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:41.189842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4d4cdb-e44d-43d4-8d83-ccab18b29b09 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:41.212236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:41.212993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:41.213351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:41.213701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.214636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:41.215048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:41.216739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:41.228379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 194/372] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:41.233480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.250912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:41.251970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:41.252470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:41.252936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.282951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:41.312350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:41.346487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cc6e6-dfd7-4f2e-b62b-c73889a17722 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:41.386505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:41.387232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:41.387719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:41.388144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.409568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:41.411221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:41.478482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:41.480669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:41.569790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 179/373] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:41.575973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.602229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:41.655198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:41.661046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 195/374] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:42:41.669013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.763941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4036e-9aaf-41d5-86ed-37435a201734 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:41.812121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:41.812885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:41.813158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:41.813424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.815756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:41.817193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:41.818826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 180/375] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:41.827003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.862154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e9f99d-493d-4434-8f85-aefa57a958d2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:41.927085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:41.928016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:41.928463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:41.928901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] PATH_INFO: `/v3/projects/4102cbba3e844357b14c2a5139b42e69` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:41.930223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] RBAC: Authorizing `identity:get_project(project_id=4102cbba3e844357b14c2a5139b42e69)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:41.931885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:41.934236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 196/376] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/projects/4102cbba3e844357b14c2a5139b42e69 => generated 344 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:41.941817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:41.986851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b74790-40e6-4584-a261-87dd6c87c715 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:42.013562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:42.032003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:42.091264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:42.091934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:42.092214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:42.092482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:42.095133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:42.097433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f887b-cc0e-4c47-aa5f-2cacedb0ec6a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:42.099903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:42.102623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 181/377] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:42.109269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:42.169458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:42.170774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:42.172188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:42.172687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] PATH_INFO: `/v3/groups/d0fac0d7fae54919b5c974ced121ab29` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:42.188216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] RBAC: Authorizing `identity:get_group(group_id=d0fac0d7fae54919b5c974ced121ab29)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:42.190001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:42.191899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 197/378] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:41 2022] GET /identity/v3/groups/d0fac0d7fae54919b5c974ced121ab29 => generated 229 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:42.197912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:42.343858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726058b-08fe-4e95-9f94-9f81a5050d23 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:42.349443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0913e309-43bd-45a0-867e-079e8a17e925 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:42.379968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:42.384179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:42.386281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:42.386604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:42.386882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:42.387652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] RBAC: Adding query filter params (group.id=d0fac0d7fae54919b5c974ced121ab29, role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=4102cbba3e844357b14c2a5139b42e69) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:42.387965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:42.389418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a54d7ea-ad8c-4c59-b9fd-a8449ecec0fc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:42.404145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 198/379] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:42 2022] GET /identity/v3/role_assignments?group.id=d0fac0d7fae54919b5c974ced121ab29&scope.project.id=4102cbba3e844357b14c2a5139b42e69&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 260 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:42.427180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:42.428121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:42.428566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:42.428988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:42.429900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=4d929fefaf4848f089f0e80abc945d88) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:42.430365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:42.432104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d731361-c265-46e3-b64d-6e145ad0c2fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:42.444002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 182/380] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:42 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 260 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:42.747126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaca71-b2a1-4bc3-8b23-bbd8e99ae59a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:42.747707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaca71-b2a1-4bc3-8b23-bbd8e99ae59a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:42.748136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccaca71-b2a1-4bc3-8b23-bbd8e99ae59a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:42.749633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 199/381] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:42.755208 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:42.755706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:42.756280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:42.784685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:42.786181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:42.841591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:42.870780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 183/382] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:43.246576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaca71-b2a1-4bc3-8b23-bbd8e99ae59a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:43.380715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccaca71-b2a1-4bc3-8b23-bbd8e99ae59a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:43.423140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1998c10-3615-424a-a552-3928b7c57a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:43.423736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1998c10-3615-424a-a552-3928b7c57a79 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.424020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1998c10-3615-424a-a552-3928b7c57a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.424464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1998c10-3615-424a-a552-3928b7c57a79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.435898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1998c10-3615-424a-a552-3928b7c57a79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c1998c10-3615-424a-a552-3928b7c57a79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:43.444967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:43.445957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:43.446801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 200/383] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:43.454593 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:43.503188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfbae-52df-4075-a88f-e366b97cc41a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.503739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfbae-52df-4075-a88f-e366b97cc41a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.504179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bfbae-52df-4075-a88f-e366b97cc41a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.506524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 201/384] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:43.525092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.525700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.526732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.559077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:43.560695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:43.591830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc3af63-4a5e-415c-8df3-e77d880b74f3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:43.594937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:43.638491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:43.640253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.643691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.644221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.645097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:43.645603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:43.647346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:43.661011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 184/385] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:43.666362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:43.671793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 202/386] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:43.724444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:43.819491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2586764b-5943-4133-99ef-fd72b342f66f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:43.866820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22faf548-d5d3-4458-b3c2-4e49c660ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:43.867459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf548-d5d3-4458-b3c2-4e49c660ec11 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.867747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf548-d5d3-4458-b3c2-4e49c660ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.868016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22faf548-d5d3-4458-b3c2-4e49c660ec11 None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.869122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faf548-d5d3-4458-b3c2-4e49c660ec11 None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:43.870600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22faf548-d5d3-4458-b3c2-4e49c660ec11 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:43.872475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 185/387] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:43 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:43.877894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:43.915566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51020d-7d53-4d91-8dd0-5fd2faba16a4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.915907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51020d-7d53-4d91-8dd0-5fd2faba16a4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.916231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b51020d-7d53-4d91-8dd0-5fd2faba16a4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.918072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 186/388] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:43.922889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:43.923166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:43.923431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:43.947111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:43.948870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:44.031818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e057f596-54b9-4b1a-be86-eea00f35a101 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.052504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 187/389] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:44.053373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.088234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.089017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.089391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.089668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.091693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:44.092393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:44.094511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:44.100296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 203/390] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:43 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:44.107790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.172021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762afdb-f013-47e1-a7c6-391b0d5ab605 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.175882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:44.208382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.209024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.209460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.209739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.220651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:44.225892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:44.226810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:44.227700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 188/391] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:44.238856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.345311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95661a-1950-4324-9e55-fde4995c4f4b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.391269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.392424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.392924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.393402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.394384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:44.397378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] RBAC: Authorizing `identity:create_grant(project_id=b378b246cf9b49a9b0410534f7cc6d92, group_id=5312f58afc5a4b4cb5c2b27a319090d2, role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:44.399086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:44.401841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592d18a-063d-4753-9c4d-68f7faf3072d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.417876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 204/392] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 15:42:44 2022] PUT /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/0cc328004b4e4f44905d38cb222157a9 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:44.419014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.474850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.476274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.476778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.478506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.479530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:44.480459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:44.483594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:44.506116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 189/393] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:44.514627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.623963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:44.645197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ef433-f9ea-470c-a5c5-d9a95d8e5422 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.676885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f648676-6206-4231-8a08-b7bbb3826b59 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.684691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.685712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.686195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.686665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.700739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:44.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:44.707444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:44.708315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:44.710731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 205/394] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:44.714243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.714881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.715158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.715427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.718087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:44.718984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.719602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:44.721442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 190/395] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:44.731981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.769106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:44.836731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cf65b-0b18-4246-8612-a6dc879906fb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.855564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d517df4-4471-446f-87f7-e5e62851fb73 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:44.875575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.876677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.877218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.877692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.878611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:44.879203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:44.880987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:44.890514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:44.891143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:44.891420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:44.891723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:44.892529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 206/396] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:44.892880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:44.894388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:44.896130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 191/397] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:44.902662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.902889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:44.913213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:45.002056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:45.011747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10298778-eb24-4097-bb18-f8e1d7aa205a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.020800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189dbb9-6473-461f-ab64-e04b96115fc2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.049327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:45.050164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.050862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.050862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.053176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d929fefaf4848f089f0e80abc945d88, group_id=5312f58afc5a4b4cb5c2b27a319090d2, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:45.054870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:45.064330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc828e3-2b69-4855-99ec-b5c1d11d2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:45.065568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc828e3-2b69-4855-99ec-b5c1d11d2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.066044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc828e3-2b69-4855-99ec-b5c1d11d2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.066949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc828e3-2b69-4855-99ec-b5c1d11d2cc5 None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.068673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc828e3-2b69-4855-99ec-b5c1d11d2cc5 None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:45.070411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc828e3-2b69-4855-99ec-b5c1d11d2cc5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:45.071154 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 192/398] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 15:42:44 2022] PUT /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:45.072570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 207/399] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:44 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:45.084360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:45.110646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb175b4-2a42-4c1a-a25d-57e09925cf27 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.111596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb175b4-2a42-4c1a-a25d-57e09925cf27 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.112737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb175b4-2a42-4c1a-a25d-57e09925cf27 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.114857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 208/400] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:45.121135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7279a08-8a1f-4d7a-8fed-e2313cc5c09e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.121425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7279a08-8a1f-4d7a-8fed-e2313cc5c09e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.121689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7279a08-8a1f-4d7a-8fed-e2313cc5c09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.164783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f7279a08-8a1f-4d7a-8fed-e2313cc5c09e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:45.166040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f7279a08-8a1f-4d7a-8fed-e2313cc5c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:45.233101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca01c6-8d51-4966-8fb1-19c65b12fad9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.247968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 209/401] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:45.249472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91373719-df9b-4535-979c-678146350b7c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.249761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91373719-df9b-4535-979c-678146350b7c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.250037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91373719-df9b-4535-979c-678146350b7c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.255892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 210/402] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:45.257274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.257566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.257842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.265923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:45.271892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:45.272586 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.273235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.273522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.274688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:45.276215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:45.278200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 193/403] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:45 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:45.279704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:45.280922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:45.288734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:45.319890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:45.377421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 211/404] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:45.414720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dabf79-d0b8-4eff-bba8-93f733623b20 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.450478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:45.451415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.451867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.452287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.454380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b378b246cf9b49a9b0410534f7cc6d92, group_id=5312f58afc5a4b4cb5c2b27a319090d2, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:45.456021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:45.473241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 194/405] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 15:42:45 2022] PUT /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:45.558642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:45.724056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef314732-7fa1-4d0c-909b-3427c5cad36c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.766657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:45.767303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.767605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.767887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.769748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:45.778967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:45.780549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:45.785442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:45.786349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:45.787181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 212/406] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:45.794422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:45.925966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:45.945701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e616c9-3d91-4f3c-b630-7d47a83af19d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.985010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:45.985551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:45.985831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:45.986101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:45.986835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:45.987138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:45.988828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:45.990372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410d9e79-3940-4f4c-ab68-3e7529341bb5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:45.999381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 213/407] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:46.005358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.070125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:46.099150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.099824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.100102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.100370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.118993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:46.125363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:46.126263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:46.127100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 195/408] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:46.132616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.150003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1263a9-21c1-4ff5-a96a-7f5fc238b8bd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:46.196501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.197591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.198614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.199085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] PATH_INFO: `/v3/groups/d0fac0d7fae54919b5c974ced121ab29` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.200608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] RBAC: Authorizing `identity:get_group(group_id=d0fac0d7fae54919b5c974ced121ab29)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:46.202475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:46.204496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 214/409] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:46 2022] GET /identity/v3/groups/d0fac0d7fae54919b5c974ced121ab29 => generated 229 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:46.211870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.271235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d52cc1-820f-411a-bdab-74faff42f39b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:46.339889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.340502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.340780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.341052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.341716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:46.342022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:46.343411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:46.359728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 196/410] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:46.365585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.370383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:46.414110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095601e6-9ec2-4b61-a36d-502378df5a56 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:46.460108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:46.481511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.482161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.482442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.482714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] PATH_INFO: `/v3/projects/4102cbba3e844357b14c2a5139b42e69` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.484123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] RBAC: Authorizing `identity:get_project(project_id=4102cbba3e844357b14c2a5139b42e69)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:46.485572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:46.487347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 215/411] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:46 2022] GET /identity/v3/projects/4102cbba3e844357b14c2a5139b42e69 => generated 344 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:46.493126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.577072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e32f100-5fe0-476d-a1ad-ebc85e0bd047 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:46.641272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.642033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.642367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.642715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.644058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:46.651798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:46.651798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 197/412] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:46 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:46.653385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.656948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f8dfe-6811-40b1-8714-ed13c25b1b6e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:46.698028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.700148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.700794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.701910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] PATH_INFO: `/v3/projects/4102cbba3e844357b14c2a5139b42e69/groups/d0fac0d7fae54919b5c974ced121ab29/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.706941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] RBAC: Authorizing `identity:create_grant(project_id=4102cbba3e844357b14c2a5139b42e69, group_id=d0fac0d7fae54919b5c974ced121ab29, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:46.708637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:46.730393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 216/413] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 15:42:46 2022] PUT /identity/v3/projects/4102cbba3e844357b14c2a5139b42e69/groups/d0fac0d7fae54919b5c974ced121ab29/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:46.799635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:46.841657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc8312-93df-4b93-905f-1de345700718 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:46.892439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c7723-134d-40eb-b951-1a7a135af017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:46.893438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7723-134d-40eb-b951-1a7a135af017 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:46.893893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7723-134d-40eb-b951-1a7a135af017 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:46.894357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c7723-134d-40eb-b951-1a7a135af017 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:46.895839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7723-134d-40eb-b951-1a7a135af017 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:46.897542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c7723-134d-40eb-b951-1a7a135af017 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:46.899354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 198/414] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:46 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:46.905440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:46.978153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:47.043691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555590e-c878-46cc-9602-916cd5208f60 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:47.080147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6d278a-45c9-4e61-8b64-4d3d79a14d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:47.080827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d278a-45c9-4e61-8b64-4d3d79a14d3a None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.081108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d278a-45c9-4e61-8b64-4d3d79a14d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.081387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6d278a-45c9-4e61-8b64-4d3d79a14d3a None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.083695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d278a-45c9-4e61-8b64-4d3d79a14d3a None admin] RBAC: Authorizing `identity:create_grant(project_id=4d929fefaf4848f089f0e80abc945d88, group_id=5312f58afc5a4b4cb5c2b27a319090d2, role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:47.085181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6d278a-45c9-4e61-8b64-4d3d79a14d3a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:47.103408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 217/415] 10.222.0.17 () {58 vars in 1442 bytes} [Wed Sep 7 15:42:46 2022] PUT /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88/groups/5312f58afc5a4b4cb5c2b27a319090d2/roles/0cc328004b4e4f44905d38cb222157a9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:42:47.137130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbfba-69b3-48d1-b95a-264398d88dcf None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.137455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbfba-69b3-48d1-b95a-264398d88dcf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.137693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345dbfba-69b3-48d1-b95a-264398d88dcf None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.139363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 199/416] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:47.145151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.145447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.145715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.165603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:47.166897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:47.246304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 218/417] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:47.528395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dbfba-69b3-48d1-b95a-264398d88dcf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:47.645834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345dbfba-69b3-48d1-b95a-264398d88dcf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:47.685645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5705819-c481-47d4-9acf-3d09eee22ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:47.686286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705819-c481-47d4-9acf-3d09eee22ded None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.686566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705819-c481-47d4-9acf-3d09eee22ded None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.686836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5705819-c481-47d4-9acf-3d09eee22ded None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.694190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:47.700423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5705819-c481-47d4-9acf-3d09eee22ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e5705819-c481-47d4-9acf-3d09eee22ded None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:47.705406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:47.706317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:47.707125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 200/418] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:47.712656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:47.757839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:47.776435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae128f6-aba6-4a13-8651-472d3f83c87b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.776755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae128f6-aba6-4a13-8651-472d3f83c87b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.777133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae128f6-aba6-4a13-8651-472d3f83c87b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.778741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 201/419] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:47.783854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f62aa-40a4-4ebf-8e07-727e3899af92 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.784133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f62aa-40a4-4ebf-8e07-727e3899af92 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.784402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f62aa-40a4-4ebf-8e07-727e3899af92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.824386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b9288-5192-4c1d-b306-59a7369fbd3e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:47.832581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-207f62aa-40a4-4ebf-8e07-727e3899af92 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:47.833800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-207f62aa-40a4-4ebf-8e07-727e3899af92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:47.860297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:47.861023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:47.861308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:47.861577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:47.862246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:47.862550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:47.864026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:47.872682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 219/420] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:47.878257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:47.986631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 202/421] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:48.029184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd0658-1346-4592-8f77-45a24f32f1fb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.043974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:48.071378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.072033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.072690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.073034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.134974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943398-0d57-477b-ae26-243c144f2509 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.135666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943398-0d57-477b-ae26-243c144f2509 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.136146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a943398-0d57-477b-ae26-243c144f2509 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.137938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 203/422] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:48.143451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.144035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.144523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.174786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:48.176353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:48.187601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:48.189070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:48.306956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 204/423] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:48.308540 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.321083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 220/424] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:48.328069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.332105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:48.399828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:48.452476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0799c5d4-4a06-43c9-8997-be82da38cce2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.491639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.492269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.492545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.492819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.495353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db0748-3ab8-45a3-8f31-28480b370145 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.511178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:48.515967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:48.516879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:48.517701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 205/425] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:48.523299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.553513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.554151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.554426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.554695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.556639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:48.558258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:48.559841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 221/426] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:48.566619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.659229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:48.696295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b5ab18-4043-4caf-99ca-38f0914ab0a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.724876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456ba4ac-ea1a-4c29-925e-1a7f487d48b4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.733057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.733694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.733972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.734243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.735617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:48.737058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:48.743424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 222/427] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:48.744297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.746032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:48.773241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.774100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.774783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.775063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.776546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:48.776913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:48.779279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:48.791013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 206/428] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:48.791960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.863954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07e818-8bc5-4519-840d-9c1be305b69a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.899424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93392f-6f91-4df2-9a44-efb39c7ef1ad None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:48.908628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.909615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.909894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.910166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.919816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:48.925828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:48.932926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:48.934683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 07 15:42:48.935802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:48.943573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 223/429] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:48.943573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:48.943765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:48.943765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:48.943765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:48.943765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:48.944096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:48.944096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:48.945027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:48.951914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 207/430] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:48.960872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.061474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.082744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbc59d-f4be-4da3-ae33-ac150fd82cc8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.122311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.122855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.123133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.123405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.194745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc540a8-5574-4875-8730-61637d0b8bdb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.199158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.202406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.204761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.238268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.238885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.239163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.239432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.240130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:49.240432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.241993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.251967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 208/431] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:49.257473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.321986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 224/432] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:42:49.329280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.388492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6e62a-626f-44f5-b467-06b57e2862ce None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.397793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.436104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.436725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.437560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.437844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.453667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.461447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cb9d0f-ac61-43f8-8fcd-f391cf471398 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.501646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.502284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.502844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.504122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.505676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.511576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 225/433] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:49 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:49.514019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.516692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.518751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.630560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108ce8f-edfc-45d6-88d8-06eb53816878 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.634499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 209/434] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:42:49.635664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.644188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.671042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d00233-b16e-4a86-8db3-4895f600da3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.671705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d00233-b16e-4a86-8db3-4895f600da3b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.671992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d00233-b16e-4a86-8db3-4895f600da3b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.672263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d00233-b16e-4a86-8db3-4895f600da3b None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.673455 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d00233-b16e-4a86-8db3-4895f600da3b None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.674880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d00233-b16e-4a86-8db3-4895f600da3b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.681147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 226/435] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:49 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:49.681147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e08972-2270-4085-8f01-1960f48522fc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.681147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e08972-2270-4085-8f01-1960f48522fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.681147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e08972-2270-4085-8f01-1960f48522fc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.681147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 227/436] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:49.682085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e08972-2270-4085-8f01-1960f48522fc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.725596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.763639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26aebbf-10a5-40c2-8947-90ef5b3cb0ec None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.796866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e08972-2270-4085-8f01-1960f48522fc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:49.801800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e7699-efe2-4004-88ff-8510adf7cf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.802461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e7699-efe2-4004-88ff-8510adf7cf7f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.802741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e7699-efe2-4004-88ff-8510adf7cf7f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.803013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e7699-efe2-4004-88ff-8510adf7cf7f None admin] PATH_INFO: `/v3/projects/b378b246cf9b49a9b0410534f7cc6d92` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.804161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e7699-efe2-4004-88ff-8510adf7cf7f None admin] RBAC: Authorizing `identity:get_project(project_id=b378b246cf9b49a9b0410534f7cc6d92)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.805590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e7699-efe2-4004-88ff-8510adf7cf7f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.812461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 210/437] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:49 2022] GET /identity/v3/projects/b378b246cf9b49a9b0410534f7cc6d92 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:49.812461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e3208-a350-431e-9992-9e0ebe2c05d2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.812461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e3208-a350-431e-9992-9e0ebe2c05d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.812461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e3208-a350-431e-9992-9e0ebe2c05d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.829038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1e9e3208-a350-431e-9992-9e0ebe2c05d2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:49.830226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e9e3208-a350-431e-9992-9e0ebe2c05d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:49.832465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:49.833055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.833332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.833627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.834326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=4d929fefaf4848f089f0e80abc945d88) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:49.834633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:49.836086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:49.841900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 228/438] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:49 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:49.842588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:49.917313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 211/439] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:49.919383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b211c-3b4c-4d49-821e-c233fff113a8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.919703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b211c-3b4c-4d49-821e-c233fff113a8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.919976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b211c-3b4c-4d49-821e-c233fff113a8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.921464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 212/440] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:49.927146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:49.927427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:49.927725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:49.935775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.951273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:49.952609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:49.964615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:49.969511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685c178-3e1f-40f2-8c38-920033bc9a62 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.008208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.008846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.009124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.009393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.011041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:50.012821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:50.014358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 229/441] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:49 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:50.020839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.033935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 213/442] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:50.122267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579e2d0-9ced-40f9-b568-89bed8f46017 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.158745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.158745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.158745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.158745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.159138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=09d9b803d29e48b28dfc0fcb6d976e31, scope.project.id=b378b246cf9b49a9b0410534f7cc6d92) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:50.159138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:50.160460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:50.166762 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 230/443] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=b378b246cf9b49a9b0410534f7cc6d92&role.id=09d9b803d29e48b28dfc0fcb6d976e31 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:50.201235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.308394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.310848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281e040-00b8-4627-8d4d-64aa6bbf6188 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.344502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:50.352993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.353630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.353908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.354233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.357442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:50.366317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:50.371750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:50.372645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:50.373456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 214/444] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:50.378744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.431196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6653e05-66d6-4271-8502-333c0da1f195 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.476325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.477492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.478072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.478524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.493038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:50.499170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205636-7c4f-48b6-adce-bfcad93a3115 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:50.499681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:50.503987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:50.504811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 231/445] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:50.509072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.545931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.545931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.545931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.545931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.546328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:50.546433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:50.547869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:50.557611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 215/446] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:50.562874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.619346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:50.629753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ecf0bb-969e-45b0-80c6-77670fd231bc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.633135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:50.675640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.676264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.676547 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.676823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.677501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:50.677809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:50.679204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:50.689311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 232/447] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:50.693708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b3caf-87cb-42b0-85c9-818083782377 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.695223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.739348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.739348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.739686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.739794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.816866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:50.818318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:50.824174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d176b-f8b4-4324-9b6a-89d273e1eb95 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:50.863732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:50.864352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:50.864633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:50.864909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:50.881527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:50.889286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:50.926365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 216/448] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:42:50.931970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:50.932839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:50.934304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.030178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad42acc-33b2-4aed-a58e-5d93ac6cac7e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:51.031387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 233/449] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:42:51.037352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:51.066677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:51.067296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:51.067590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:51.067864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] PATH_INFO: `/v3/projects/4102cbba3e844357b14c2a5139b42e69` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:51.068954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] RBAC: Authorizing `identity:get_project(project_id=4102cbba3e844357b14c2a5139b42e69)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:51.070367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.072223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 217/450] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:50 2022] GET /identity/v3/projects/4102cbba3e844357b14c2a5139b42e69 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:51.077357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:51.095614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:51.101414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:51.139825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41cae6-366c-4348-be84-f531a2e066d0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:51.174458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83e12-b40a-4ae1-a573-feb9dcd058b5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:51.175859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:51.176522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:51.176802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:51.177071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] PATH_INFO: `/v3/projects/4d929fefaf4848f089f0e80abc945d88` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:51.178173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] RBAC: Authorizing `identity:get_project(project_id=4d929fefaf4848f089f0e80abc945d88)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:51.179679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.181287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 234/451] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:42:51 2022] GET /identity/v3/projects/4d929fefaf4848f089f0e80abc945d88 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:51.186758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:51.207581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:51.208215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:51.208493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:51.208764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] PATH_INFO: `/v3/groups/d0fac0d7fae54919b5c974ced121ab29` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:51.209873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] RBAC: Authorizing `identity:get_group(group_id=d0fac0d7fae54919b5c974ced121ab29)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:51.211296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.212875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 218/452] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:51 2022] GET /identity/v3/groups/d0fac0d7fae54919b5c974ced121ab29 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:51.218947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:51.284840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771179c5-e1df-4cbf-9a54-b0c06e51aca3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:51.306674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:51.307226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:51.316585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94990a8-2c12-4a03-a1c1-898eff7ad96c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:51.318685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:51.319312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:51.319613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:51.319887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] PATH_INFO: `/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:51.320989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] RBAC: Authorizing `identity:get_group(group_id=5312f58afc5a4b4cb5c2b27a319090d2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:51.322421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.324886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 235/453] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Sep 7 15:42:51 2022] GET /identity/v3/groups/5312f58afc5a4b4cb5c2b27a319090d2 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:51.330524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:51.350495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:51.351084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:51.351359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:51.351984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:51.352368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] RBAC: Adding query filter params (group.id=d0fac0d7fae54919b5c974ced121ab29, role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=4102cbba3e844357b14c2a5139b42e69) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:51.352669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:51.354664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a4875-f290-4f76-ae0b-67d29be3400c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.371696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 219/454] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:51 2022] GET /identity/v3/role_assignments?group.id=d0fac0d7fae54919b5c974ced121ab29&scope.project.id=4102cbba3e844357b14c2a5139b42e69&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:51.434133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430ad44-8e1e-4536-9526-682caede270e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:51.468396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:51.469025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:51.469309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:51.469583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:51.470324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] RBAC: Adding query filter params (group.id=5312f58afc5a4b4cb5c2b27a319090d2, role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=4d929fefaf4848f089f0e80abc945d88) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:51.470632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:51.472137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc9fa2c-5c32-4452-8eea-d4ee9fc5c75c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:51.478466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 236/455] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Sep 7 15:42:51 2022] GET /identity/v3/role_assignments?group.id=5312f58afc5a4b4cb5c2b27a319090d2&scope.project.id=4d929fefaf4848f089f0e80abc945d88&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:54.087270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a89f7-af9d-4dc3-8850-03b935f9ec0f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.087809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a89f7-af9d-4dc3-8850-03b935f9ec0f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.087887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a89f7-af9d-4dc3-8850-03b935f9ec0f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.089929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 220/456] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:54.095208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fcfe2-7d60-4e5d-9105-63f4e4e37268 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.096022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fcfe2-7d60-4e5d-9105-63f4e4e37268 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.096454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fcfe2-7d60-4e5d-9105-63f4e4e37268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.122070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f67fcfe2-7d60-4e5d-9105-63f4e4e37268 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:54.123573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f67fcfe2-7d60-4e5d-9105-63f4e4e37268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:54.189077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:54.212222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 237/457] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:54.334993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41783-adc2-4203-8f77-9cfa74bc4b6d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.335654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41783-adc2-4203-8f77-9cfa74bc4b6d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.336132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f41783-adc2-4203-8f77-9cfa74bc4b6d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.338009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 221/458] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:54.342900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4733c-0b5e-4a8d-9e73-b094468b160f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.343196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4733c-0b5e-4a8d-9e73-b094468b160f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.343477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e4733c-0b5e-4a8d-9e73-b094468b160f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.368515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-26e4733c-0b5e-4a8d-9e73-b094468b160f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:54.370276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26e4733c-0b5e-4a8d-9e73-b094468b160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:54.465479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 238/459] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:54.548793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41783-adc2-4203-8f77-9cfa74bc4b6d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:54.557608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba2731-3651-40ba-8ea5-0590a3227efd None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.557897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba2731-3651-40ba-8ea5-0590a3227efd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.558177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba2731-3651-40ba-8ea5-0590a3227efd None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.559578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 239/460] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:54.564953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.565255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.565534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.593735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:54.594837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:54.692520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f41783-adc2-4203-8f77-9cfa74bc4b6d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:54.732346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 240/461] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:42:54.754997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:54.756010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.756517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.756996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.775462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:54.776988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:42:54.800097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:54.801062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:42:54.801937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:42:54.801937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:54.801937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:54.801937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 222/462] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:54.809231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:54.853923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:54.865963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:54.880545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:54.943062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c05857-9c12-4e0e-ac21-b2a085e0bbf4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:54.981873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:54.982501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:54.982784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:54.983059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:54.983734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:54.984044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:54.985432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.019931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd925a7-df3e-4aa9-9218-71d11884f23f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.025776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 241/463] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:55.031179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:55.059788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.060422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.060757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.060994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.073296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.075070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:42:55.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:55.092417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:42:55.093402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:42:55.093402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:55.093402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:55.095089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 223/464] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:55.100609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:55.173529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:55.182650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c35cc-1f79-4259-a1c5-9c6b022d38d8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.187717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:55.223662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa3599d-9af0-4313-aa13-02936d49427b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.239324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.240786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.241067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.241335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.242005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.243455 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.268990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.269889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.270209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.270519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.283785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 242/465] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:55 2022] POST /identity/v3/users => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:42:55.285146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:55.293558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.295998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a99a75b2-3a74-4e92-afcf-2231789f2816 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:42:55.320809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:55.321814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:42:55.323238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:42:55.323238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:55.323238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:55.323238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 224/466] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:55.328307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ad1f-bc81-4c61-93f3-3214335aa230 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.329876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ad1f-bc81-4c61-93f3-3214335aa230 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.330382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ad1f-bc81-4c61-93f3-3214335aa230 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.331890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 225/467] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:55 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:55.332776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ad1f-bc81-4c61-93f3-3214335aa230 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:55.416058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7daa7a-bea1-42a1-be9a-c4fdfecf6f49 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.454776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ad1f-bc81-4c61-93f3-3214335aa230 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.455292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.456189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.456676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.457176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.458042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:55.458566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.460250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da812a64-910c-41ad-92d0-305d54a51af4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.470299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:55.473964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 243/468] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:55.480165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.480876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.481375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.495774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20614ef-b171-489f-a312-d7436639171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.497836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20614ef-b171-489f-a312-d7436639171c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.498577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20614ef-b171-489f-a312-d7436639171c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.499356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20614ef-b171-489f-a312-d7436639171c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.500608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20614ef-b171-489f-a312-d7436639171c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:55.500909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20614ef-b171-489f-a312-d7436639171c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.502759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20614ef-b171-489f-a312-d7436639171c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.503066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:55.504706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:55.517646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 226/469] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:55.518817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20614ef-b171-489f-a312-d7436639171c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:55.527990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:55.599931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 244/470] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:55.604412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:55.643015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20614ef-b171-489f-a312-d7436639171c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.680129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415902f5-e9f6-4c01-9c0d-d63fa1a8c64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.680655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415902f5-e9f6-4c01-9c0d-d63fa1a8c64e None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.680933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415902f5-e9f6-4c01-9c0d-d63fa1a8c64e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.681203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415902f5-e9f6-4c01-9c0d-d63fa1a8c64e None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.681852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415902f5-e9f6-4c01-9c0d-d63fa1a8c64e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.683827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415902f5-e9f6-4c01-9c0d-d63fa1a8c64e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.707368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 227/471] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:55 2022] POST /identity/v3/users => generated 248 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:55.812189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f42e0-0a6d-4495-8d12-cec53e2b1c11 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:55.873477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be68d2-26d8-41f5-b6ef-2c45abb059e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:55.874116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be68d2-26d8-41f5-b6ef-2c45abb059e4 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.874401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be68d2-26d8-41f5-b6ef-2c45abb059e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.874678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be68d2-26d8-41f5-b6ef-2c45abb059e4 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.875341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be68d2-26d8-41f5-b6ef-2c45abb059e4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:55.876864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be68d2-26d8-41f5-b6ef-2c45abb059e4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:55.908238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 245/472] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:55 2022] POST /identity/v3/users => generated 248 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:42:55.940409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172df792-d2c3-406f-af7f-3b6bbeac213a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.940649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172df792-d2c3-406f-af7f-3b6bbeac213a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.940889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172df792-d2c3-406f-af7f-3b6bbeac213a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.942554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 228/473] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:55.948985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:55.949283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:55.949541 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:55.964921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:55.972912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:55.976156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:56.057132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 246/474] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:56.057327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172df792-d2c3-406f-af7f-3b6bbeac213a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:56.096198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:56.180932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172df792-d2c3-406f-af7f-3b6bbeac213a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:56.222644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:56.223683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:56.224200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:56.224527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:56.238256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:56.239756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:42:56.255882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:56.256846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:42:56.257720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:42:56.257720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:56.257720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.257720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 229/475] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:56.263344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:56.361280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:56.380859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b24718-43f4-4c68-a2df-225af5b5b9ae None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:56.419089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:56.419740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:56.420019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:56.420288 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:56.420929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:56.421677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:56.423342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:56.426511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:56.439566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 247/476] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:56.444669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:56.459948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:56.488956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508e66a-c3ca-411f-8c21-0981b082e4e7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:56.550587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:56.551202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:56.551475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:56.551768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:56.552440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:56.553805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:56.570630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.sql.core [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 15:42:56.570630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 15:42:56.570630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [parameters: {'id': 'a604c3c3b2364f35bf48b6541b56bc88', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 15:42:56.570630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81458) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application conn.query(q) Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result.read() Sep 07 15:42:56.602417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return ref.to_dict() Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 15:42:56.603357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application next(self.gen) Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application yield resource Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application next(self.gen) Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise self.value Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application session.commit() Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 15:42:56.605117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.session.flush() Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application compat.raise_( Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 15:42:56.606612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 15:42:56.608835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application conn.query(q) Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application result.read() Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 15:42:56.610908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [parameters: {'id': 'a604c3c3b2364f35bf48b6541b56bc88', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:56.617050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 07 15:42:56.619108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.628740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 230/477] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 15:42:56 2022] POST /identity/v3/roles => generated 144 bytes in 244 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:56.631177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:56.642174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a290f91-fa34-40a1-83bf-3f05485fef54 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:56.687701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:56.688372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:56.688663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:56.688938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:56.689600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:56.691057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:56.735508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 248/478] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:42:56 2022] POST /identity/v3/users => generated 249 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:42:56.796924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9589cd6-ebc0-4d17-844a-5a998cecd131 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:56.841776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72573970-4018-485d-a87d-08d48fa971a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:56.842439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72573970-4018-485d-a87d-08d48fa971a8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:56.842721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72573970-4018-485d-a87d-08d48fa971a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:56.842994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72573970-4018-485d-a87d-08d48fa971a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:56.861761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72573970-4018-485d-a87d-08d48fa971a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-72573970-4018-485d-a87d-08d48fa971a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:56.866678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:56.868757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 07 15:42:56.870759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:56.872241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 231/479] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:42:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:56.878381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:56.949634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:57.016667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8183cc60-a218-490c-a3b4-afaeca141d1d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:57.067061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:57.067675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:57.067917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:57.068190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:57.068863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:57.069165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:57.070550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ce9919-b2ec-47b6-9c6e-e687bbb87181 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:57.079137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 249/480] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:42:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:57.886139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305eba-c28c-464c-a646-18ae3b6eac11 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:57.886139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305eba-c28c-464c-a646-18ae3b6eac11 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:57.886139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f305eba-c28c-464c-a646-18ae3b6eac11 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:57.886139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 232/481] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:57.893043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83aa3ad-969f-4aef-8407-cea737515ca6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:57.893559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83aa3ad-969f-4aef-8407-cea737515ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:57.893709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83aa3ad-969f-4aef-8407-cea737515ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:57.916804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b83aa3ad-969f-4aef-8407-cea737515ca6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:57.918011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b83aa3ad-969f-4aef-8407-cea737515ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:58.006409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 250/482] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:58.393833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f305eba-c28c-464c-a646-18ae3b6eac11 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:58.398254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:58.480133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d462e48-e1d7-4a54-b6d4-098ebda475ff None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:58.480431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d462e48-e1d7-4a54-b6d4-098ebda475ff None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:58.480612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d462e48-e1d7-4a54-b6d4-098ebda475ff None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:58.481959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 251/483] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:58.486958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ac7cc-5e0c-4080-9e30-8c8f845ad99b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:58.487239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ac7cc-5e0c-4080-9e30-8c8f845ad99b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:58.487507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ac7cc-5e0c-4080-9e30-8c8f845ad99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:58.516172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-352ac7cc-5e0c-4080-9e30-8c8f845ad99b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:58.517369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-352ac7cc-5e0c-4080-9e30-8c8f845ad99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:58.548811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f305eba-c28c-464c-a646-18ae3b6eac11 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:58.607975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:58.630065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:58.630707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:58.630982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:58.631250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:58.643212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:58.648476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:58.649404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:58.651181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 233/484] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:42:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:58.656793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:58.697190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 252/485] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:58.723750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:58.732047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e42f01-e010-47f3-8c55-37bee3bc65bb None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:58.732305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e42f01-e010-47f3-8c55-37bee3bc65bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:58.732511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e42f01-e010-47f3-8c55-37bee3bc65bb None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:58.734099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 253/486] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:42:58.740946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:58.741344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:58.741712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:58.761001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:58.762422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:58.778408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c33201d-f689-4d02-9ff8-cdce2f2ac555 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:58.833385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 254/487] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:58.856783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:58.857392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:58.857670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:58.857938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:58.858599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:58.858901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:58.860315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:58.869082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 234/488] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:42:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:58.874534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.013319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbc9933-c238-4685-8c3a-3add8c6464b6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.022226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.065243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.065886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.066199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.066433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.087042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.088665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.106373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:42:59.111756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:59.112628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:42:59.113502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:42:59.113502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:59.113502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:59.113502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 255/489] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:59.119274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.250609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791884bb-8858-4946-8cd3-e7e84c41cf4d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.254934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad59d67-37f0-4a09-98a1-0107e295a2c6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.277078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.286016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.286845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.287335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.287750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.293008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.293623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.293903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.294176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.294825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:59.295128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.296737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.299522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:59.304959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:59.306037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:42:59.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:59.309215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 235/490] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:42:59.310072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.310285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 256/491] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:42:59.315175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.336449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.417375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3706eb6-6055-419b-8acf-d38e33c8b7af None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.428830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176816-bc6d-4e6b-9b1e-66812f6c559f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.452205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.452960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.453277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.453605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.465670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:42:59.468469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.469373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.469838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.470291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.471172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:59.471712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:42:59.472093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:59.472969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:59.474075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 236/492] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:59.474979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.476712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.487653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 257/493] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:59.489052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.527197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.590348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.603457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d02983-ce24-46f1-a349-1c4182ca1f6b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.618652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426c004-53bf-47b5-bda3-6c161add1265 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.646907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.647518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.647832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.648103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.654407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.655220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.655674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.656089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.656506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.657056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:59.657520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.657941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.659367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.667901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 258/494] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:59.668843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2c5604a5-1d76-4a8c-9f25-8eeadc1efc4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:42:59.670735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:59.671635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:42:59.672556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:42:59.676410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 237/495] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:59.677628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0291ef-169b-4ab2-b438-719bac3b5309 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.677909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0291ef-169b-4ab2-b438-719bac3b5309 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.678180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0291ef-169b-4ab2-b438-719bac3b5309 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.679414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 238/496] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:59.680117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0291ef-169b-4ab2-b438-719bac3b5309 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.792163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2bbe02-b831-4ffb-b7b5-4edacb1fd929 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.795440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0291ef-169b-4ab2-b438-719bac3b5309 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:42:59.803272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.831249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.831918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.832442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.832442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.837141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:42:59.837742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.838020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.838420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.839146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:42:59.839574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.840996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b577c-e202-4541-999f-5a7d67c167ed None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.841990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:42:59.845129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:42:59.848473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:42:59.863802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:42:59.864726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:42:59.865566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:42:59.865566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:42:59.865566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:42:59.865566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 259/497] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:42:59.865800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.878191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 239/498] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:42:59.879407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f83d0d-01cd-4ef6-b070-7b41e1f550a3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.879722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f83d0d-01cd-4ef6-b070-7b41e1f550a3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.879995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f83d0d-01cd-4ef6-b070-7b41e1f550a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.898775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e6f83d0d-01cd-4ef6-b070-7b41e1f550a3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:42:59.901383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6f83d0d-01cd-4ef6-b070-7b41e1f550a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:42:59.969501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 240/499] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:42:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:42:59.970765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e006d4-9404-4843-be14-4f308e5337a5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:42:59.971058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e006d4-9404-4843-be14-4f308e5337a5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:42:59.971334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e006d4-9404-4843-be14-4f308e5337a5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:42:59.972607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 241/500] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:42:59.973298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e006d4-9404-4843-be14-4f308e5337a5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:42:59.982838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f52f4d-791a-4599-ad6f-98e62cf4d728 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.018662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.019411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.019621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.020039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.030743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.031910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:00.045962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:00.046852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:00.048124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:00.048124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:00.048124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:00.048124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 260/501] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:00.048124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.080556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.090374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e006d4-9404-4843-be14-4f308e5337a5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.099261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.126296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.126915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.127453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.127750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.128388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:00.128689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.130051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd55ab9-83c9-4716-9a5e-d6f68e405c32 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.146190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 242/502] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:00.146190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.146190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.146190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.164507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb931bf-db20-49a8-8d99-3f5e48ecc1d6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.164648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:00.165783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:00.199557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.200269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.200557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.201010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.242768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 243/503] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:00.243634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.282245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.283724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.328737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.337224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.357989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5172c-e1c6-4671-ab69-10a663780e7d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.396538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.397150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.402126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.402248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 261/504] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:00.402340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.402340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:00.402518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.402627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.404047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.424175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 244/505] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:00.424175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.508482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46f89ae-3ab1-4cf6-8ad1-5510e9898fdf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.521989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45056b8-bf68-4801-b3df-bf71117601ab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.543126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.543777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.544072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.544337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.551154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.552645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.555740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.556367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.556649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.556987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:00.565822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:00.566872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 262/506] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:00.567813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.572404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:00.575075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:00.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:00.578655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:00.579518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 245/507] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:00.581345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.583763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.670263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae613120-d2d9-4cf5-926a-4343e62f8e43 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.678458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac20ff44-2515-4e91-a843-09fb0f8f3cb9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.703165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.703829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.704117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.704396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.711530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.712167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.712444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.712720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] PATH_INFO: `/v3/users/glance` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.715204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.716706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.720598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.722007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:00.731505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:00.732406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:00.733291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:00.733291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:00.733291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:00.733291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 263/508] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:00.733509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:00.733980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:00.737968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 246/509] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:00.739077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.792947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.799486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:00.840337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f8d2b-3086-4e9b-8899-692b505251de None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.840513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e456d8-f913-40c3-a0bf-ff0a7137670c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:00.873686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.874037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:00.874279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.874556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.874669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:00.874825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.874915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:00.875182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:00.875537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:00.875866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.877293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.886468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:00.888093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:00.894700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 247/510] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:00.895655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:00.907377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:00.908358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:00.909266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:00.909266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:00.909266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:00.909266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 264/511] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:00.909266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:00.995452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5adfb-fbcd-4932-8608-699bd9cf64e3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.007795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f71a4-bb55-40c4-a4fc-f59106cf7ab8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.016910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.017238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.029039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.029741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.030018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.030313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.030962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:01.031263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.032655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.040390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 248/512] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:01.041674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.042575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.043184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.043465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.043769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.044414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:01.044722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.046090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.057710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 265/513] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:01.058496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.140081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c47b25-bb1d-43b0-8bcf-52d9659f4da7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.154788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188de798-bf57-4ae2-b57b-b4dffd644e9b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.175407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.176050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.176330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.176601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.177380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:01.177623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.179099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.188343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.188951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.189234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.189508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.196002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 249/514] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:01.197140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.224737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.241337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.258314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.259826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.296980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f052cd78-cf94-4a40-b781-5ed187311849 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.330148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.330752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.331032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.331307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.331971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:01.332278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.333624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.344604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 250/515] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:01.345386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.364637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 266/516] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:01.364637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.428745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.443424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5dd85-da71-43af-9b18-4da1c73cb8e1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.444987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.461983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764bf2d6-c0b1-4ed3-a68a-2ff066e661a8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.477613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.478233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.478518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.478792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.489330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.490762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.495497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.496153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.496436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.496711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.503497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.504963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:01.505062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:01.506094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:01.507151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:01.507151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:01.507151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:01.507151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 251/517] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:01.507151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:01.519640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:01.520695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 267/518] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:01.521810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.609251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896a83f-3669-4487-8ac4-1d827624fae4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.621992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3f8a-4ce6-4f8c-852f-a9e2d40ed316 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.641111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.641709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.641984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.642251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.647177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.656278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.656874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.657156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.657425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.658059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.659501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.666822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.680084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 268/519] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:43:01 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:01.680808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.725995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.727503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.812858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df933b-6fe4-4f13-9620-bddb17399a27 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:01.850707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:01.851363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:01.851686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:01.851968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:01.856967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 252/520] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:01.858623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.866301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:01.867916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:01.883764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:01.884688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:01.886098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:01.886098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:01.886098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:01.886098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 269/521] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:01.886332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:01.926498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.927845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:01.976657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675b968-5ffd-48c7-a55f-986398066774 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.001315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f25d76-2fdf-4cde-b718-d92aad10997c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.013758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.014363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.014636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.014906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.015574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.015869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.017325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.032767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 253/522] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.032767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.044318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.044935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.045220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.045529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.046198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.046511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.048005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.071221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 270/523] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.071221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.154279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a0612d-0c92-4421-8c95-c872e3b05a29 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.172711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.176421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.184756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d6fd5a-d6f6-4f18-ac05-052f4f348dff None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.195780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.196414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.196695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.196964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.208834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.210705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.224832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:02.225426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:02.226334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:02.227231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:02.227231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:02.227231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:02.227408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.227408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.227408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.228198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 254/524] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:02.228198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.232090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.232090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.233086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.249691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 271/525] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.249691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.338089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4f9a1-17b8-4577-ba24-5db290ac823f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.352400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66fbaaa-80c9-4509-b6a5-48650a4727b1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.376339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.377199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.377616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.378087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.386759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.387107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.387372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.387696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.387974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.388710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=8a9090ab0ffa4b7eafcc742880237229) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.389025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.389565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.390459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.395875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 272/526] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/role_assignments?user.id=8a9090ab0ffa4b7eafcc742880237229&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.396657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:02.411076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:02.412240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 15:43:02.413431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:02.421389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 255/527] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:02.422578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.430111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.464769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.533930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f39eb2e-d826-4b89-8d5f-1bfb409a768c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.570864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e3a7b-aa06-415f-bdea-d7afb361e9c0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.573354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.574153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.574581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.575023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.576964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.577471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.579066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.593136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 273/528] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.594406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.612202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.612857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.613141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.614246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.621971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.623427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:02.636922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:02.637807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:02.638658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 256/529] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:02.639124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.694173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.721136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b22d3-5e08-44b4-81ac-114034f345d9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.738615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.755699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b6c18-914d-43ac-903e-0eb8c612f4a4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.760775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.761467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.761826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.762169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.762971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.763364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.764911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.798660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.799215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.799490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.799790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.807120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.807368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 274/530] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.808289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.812069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:02.827669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:02.828581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:02.829430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:02.829848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 257/531] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:02.830380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.928399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460c875-0cc5-48fe-b057-e1bd6023090f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.954654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbf3ef-0053-49dc-9b6a-37e10fbd0690 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:02.967462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:02.968710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:02.969175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:02.969815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:02.971077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:02.971590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:02.973617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:02.983894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:02.991897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 275/532] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:02.992995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:02.999949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:03.000800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.001082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.001347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.023106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:03.092492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:03.093928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:03.121065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b24ae82-d0f9-4ef9-b94d-70f17afc00aa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:03.157126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:03.158237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.158714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.159178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.160109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:03.161242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:03.162890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:03.179952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 276/533] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:03.181109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:03.200125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 258/534] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:03.200125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:03.248708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:03.250732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:03.294504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f035427-c153-476a-9b2a-cf0b919e14b0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:03.308874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43450062-e76b-4520-856c-f45549a8d3c8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:03.330721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:03.331716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.332193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.332656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.333584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=396fc6e85a65437898bc9d964e429de3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:03.334087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:03.335752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:03.347799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 277/535] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:03 2022] GET /identity/v3/role_assignments?user.id=396fc6e85a65437898bc9d964e429de3&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:03.347799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:03.355046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:03.355582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.358413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.358413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.358413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=257da2327d944095b51df9f677b3a99c) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:03.358413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:03.359864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:03.375923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 259/536] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:03 2022] GET /identity/v3/role_assignments?user.id=257da2327d944095b51df9f677b3a99c&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:03.504067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66330b23-1e60-40fe-8698-7d48eb4c6508 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:03.531312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:03.555299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:03.556007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.556290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.556637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.569852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:03.571417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:03.589565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:03.590564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:03.591490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:03.591490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:03.591490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:03.591490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 278/537] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:03.598148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:03.729489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c2efd2-7d06-4715-8b6b-c1d5dfc69c08 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:03.745786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:03.785139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:03.786092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.786562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.787912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:03.788412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:03.789977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8b870-9979-4a46-b633-1e644ee23229 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:03.808088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 260/538] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:03.814138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:03.934342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d87ced-f2b5-4566-8da4-76de24d9a7b8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.934694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d87ced-f2b5-4566-8da4-76de24d9a7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.934959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d87ced-f2b5-4566-8da4-76de24d9a7b8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.936545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 261/539] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:03.941709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:03.941991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:03.942259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:03.958401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36b2253-a453-4a02-bee6-5192fefa6f14 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:03.960934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:03.962129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:04.026727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:04.027378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:04.027727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:04.028003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:04.038489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:04.040005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:04.063200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:04.064126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:04.065244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 279/540] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:04.070667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:04.081790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 262/541] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:43:04.120641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:04.199588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86acdbc-792b-4a45-ab7e-64b714abd526 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:04.235859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:04.236476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:04.236755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:04.237027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:04.237724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:04.238027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:04.239487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:04.252400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 280/542] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:04.257959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:04.399929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:04.404691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be06f36f-7dc8-4d24-95cf-c6f2c7df69e2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:04.454582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475c758-2237-42f9-a699-e15211115ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:04.455212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475c758-2237-42f9-a699-e15211115ede None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:04.455499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475c758-2237-42f9-a699-e15211115ede None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:04.455802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475c758-2237-42f9-a699-e15211115ede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:04.456557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475c758-2237-42f9-a699-e15211115ede None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=690506f288f84debaadb4c8975759dad) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:04.456872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475c758-2237-42f9-a699-e15211115ede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:04.458345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475c758-2237-42f9-a699-e15211115ede None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:04.468214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 263/543] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:04 2022] GET /identity/v3/role_assignments?user.id=690506f288f84debaadb4c8975759dad&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:04.502052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:04.610003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:04.617070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0673d5f7-b913-48d0-bef9-71cd8aab03a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:04.654424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:04.655084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:04.655364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:04.655693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:04.667709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:04.673485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:04.675711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:04.677657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 281/544] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:04.683583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475c758-2237-42f9-a699-e15211115ede None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:04.785651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475c758-2237-42f9-a699-e15211115ede None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:04.830646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:04.831267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:04.831569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:04.831840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:04.832515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:04.832819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:04.834203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:04.843479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 264/545] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:04.852671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:05.015214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42328e3-ed7b-4276-a87b-349dfb526b61 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:05.061405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:05.061993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.062278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.062552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.079694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:05.081296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c1a994b5-e422-437b-b239-1cfb09029042 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:05.102696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:05.105107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:05.107115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:05.107115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:05.107115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:05.111858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 282/546] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:05.117145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:05.151980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:05.234700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc122d74-77ed-4c98-b231-da6fe785ec1f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:05.265405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af60f5-647f-47b7-9bc4-dc93ad52d524 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.266251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af60f5-647f-47b7-9bc4-dc93ad52d524 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.266672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6af60f5-647f-47b7-9bc4-dc93ad52d524 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.268448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 283/547] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:05.274407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:05.276060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.276469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.276718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.277001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.277229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.277317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.278063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:05.278492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:05.280308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:05.292970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 265/548] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:05.299712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:05.304365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:05.319611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:05.320861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:05.397605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:05.411068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5019eeb4-5361-42ac-b192-164a0f9820eb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:05.447095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:05.448075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.448498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.448932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.456775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:05.458404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:05.471242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:05.472937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:05.474275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:05.476451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 266/549] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:05.482740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:05.502326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 284/550] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:05.588990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77fc4c-c623-45a9-a9be-debb1d9687d8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:05.624083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:05.624952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.625377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.625796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.626657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:05.627116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:05.628726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:05.651687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 267/551] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 15:43:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:05.660637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:05.798005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41efaff4-5261-4851-8466-b9fb055f5a02 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:05.812522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:05.824045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:05.835448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0deee6d-8ae1-46ee-9263-0ff1e8c7534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:05.836099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deee6d-8ae1-46ee-9263-0ff1e8c7534e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:05.836383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deee6d-8ae1-46ee-9263-0ff1e8c7534e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:05.836661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0deee6d-8ae1-46ee-9263-0ff1e8c7534e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:05.840483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:05.962209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48d8fb6-3d13-4042-a93b-62e84e7d8858 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:05.989991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0deee6d-8ae1-46ee-9263-0ff1e8c7534e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:05.991487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0deee6d-8ae1-46ee-9263-0ff1e8c7534e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.004299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641a439a-2fde-4b9d-8e6e-a14b178ad79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.004947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a439a-2fde-4b9d-8e6e-a14b178ad79a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.005328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a439a-2fde-4b9d-8e6e-a14b178ad79a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.005616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641a439a-2fde-4b9d-8e6e-a14b178ad79a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.017173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641a439a-2fde-4b9d-8e6e-a14b178ad79a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-641a439a-2fde-4b9d-8e6e-a14b178ad79a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:06.022611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:06.023687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:06.024533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 268/552] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:06.028349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c11889-42b5-470f-8ca3-b3c98948cb49 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.028634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c11889-42b5-470f-8ca3-b3c98948cb49 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.029028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c11889-42b5-470f-8ca3-b3c98948cb49 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.030203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 269/553] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:06.030909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c11889-42b5-470f-8ca3-b3c98948cb49 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.110077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.123306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 285/554] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:06.124645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.124934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.125207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.142784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:06.144033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:06.153173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c11889-42b5-470f-8ca3-b3c98948cb49 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.175033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.200482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.201096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.201376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.202082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.203071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:06.203395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.205343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e9b0d7-a9d6-49b5-8918-e141046286f9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.219582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 270/555] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:06.219582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5ea85-1e57-4e4c-bc11-f57f8e4ac8c5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.219582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5ea85-1e57-4e4c-bc11-f57f8e4ac8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.219582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5ea85-1e57-4e4c-bc11-f57f8e4ac8c5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.219582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 271/556] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:06.219582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5ea85-1e57-4e4c-bc11-f57f8e4ac8c5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.220043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 286/557] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:06.223648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.328429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf5ac6-ef3c-4f8e-9216-57b1b4b7d67d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.335077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5ea85-1e57-4e4c-bc11-f57f8e4ac8c5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.366001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.375537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.376199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.376478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.376750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.384215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.386152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.386696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.387735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.389280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.391234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.402116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.403781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8b4e5f79-5dc6-4d2f-80cc-55bddcaa4be7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:06.406748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:06.407846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:06.408825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:06.408825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:06.408825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:06.413299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 287/558] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:06.414281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.414787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.414941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:06.424720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:06.425713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:06.426622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:06.426622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:06.426622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:06.433363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 272/559] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:06.433363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.439013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:06.440341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:06.458169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.533590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 288/560] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:06.534846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.559001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff917f51-405d-48f7-b46b-9b0056a23b25 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.607510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.608515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.609827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.610306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.611152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:06.611684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.613263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.626326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 273/561] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:06.627521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.663604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16b884-4567-4e38-bb38-d70b2298e8f7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.670019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.708472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.709119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.709791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.710064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.710711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:06.711015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.713356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.731689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 289/562] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:06.731689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.743027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.743272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989bcb0e-8e7e-46da-b3c0-13e88971753e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.794848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.796220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.796656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.797347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.811879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:06.818502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:06.819687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:06.820762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:06.821427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 274/563] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:06.821849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.859294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863314fe-68f4-432f-bb79-ad996cefacc9 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.895088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.895674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.896304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.896304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.905981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.907566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:06.924319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:06.925357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:06.926312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:06.928158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 290/564] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:06.933145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:06.934340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385df9e-ebe3-4aca-b3a4-02b6e678ba55 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:06.934508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:06.974329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:06.975205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:06.975675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:06.976119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:06.984233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:06.985834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:06.998120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:06.999271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:07.000387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:07.002222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 275/565] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:07.003086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.012859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.053301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c4155-7c57-4ca9-8da7-0918d8168b98 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.089217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.089853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.090132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.090403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.102180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:07.108336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:07.109957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 291/566] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:07.110842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.139606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e694ee-657e-4820-b858-a62096cb47b8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.176598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.177204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.177478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.177744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.178400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:07.178699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.180095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.188518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 276/567] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:07.189285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.216659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.219354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e299cf04-d8b9-4248-afc4-027cd6a2a46b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.246204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.254197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.254806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.255088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.255355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.256117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:07.256421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.257842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.274640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 292/568] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:07.275452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.307454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be456c7f-7e31-4d27-9954-844d059bedef None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.343199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.343866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.344121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.344393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.345085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:07.345390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.346832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3f9334-1821-446a-9755-618ee7c5a75a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.365721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 277/569] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:07.367272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561f99-ad89-4f7e-ae97-d2b6019da9bf None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.367794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561f99-ad89-4f7e-ae97-d2b6019da9bf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.368276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1561f99-ad89-4f7e-ae97-d2b6019da9bf None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.370336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 278/570] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:07.372386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561f99-ad89-4f7e-ae97-d2b6019da9bf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.376540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3a59c-4f36-444c-b498-858321cfef72 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.415097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.415655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.415951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.416208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.416873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:07.417182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.418553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.426463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 293/571] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:07.427246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.460997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.477392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.485866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1561f99-ad89-4f7e-ae97-d2b6019da9bf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.520157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.520785 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.521060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.521327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.531298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.532779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.532934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec508336-60e9-4db4-93f4-060691ee8c63 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:07.550271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:07.551535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:07.552560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:07.552560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:07.552560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:07.552560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 279/572] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:07.552560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.574579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.575210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.575489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.575788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.586275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.587737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-fd63ae1e-9b24-43a8-a8fc-347dc8926e97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:07.602193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:07.603180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:07.604743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:07.604743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:07.604743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:07.604743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 294/573] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:07.605576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.605856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.606126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.624677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:07.625861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:07.656396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e01ca-e5ff-45f2-92f5-ddfb1c762cbe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.685525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.690079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.690829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.691078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.691346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.692121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=f92a1a83dc5a4ababdebb85429de82b0) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:07.692647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.694615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.696893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 295/574] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:07.697761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.700240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 280/575] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/role_assignments?user.id=f92a1a83dc5a4ababdebb85429de82b0&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:07.707607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.734621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:07.845940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa013e7-ce0b-43e9-aaa6-ccbaf4bad92d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.889614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50329a4f-9af8-46b1-9f8a-83b775bd5a93 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:07.895190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.895881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.896163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.896438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.915589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.915589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:07.948489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:07.949501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:07.950471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:07.950471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:07.950471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:07.954242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 296/576] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:07.956640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:07.958068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:07.958853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:07.961546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:07.962000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:07.962832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:07.963854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:07.966285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:07.990999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 281/577] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:07.994301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.060193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.081813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.102477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10b58c-bc8e-44e5-9829-b498d2ef407f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.115981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b79d1-5c81-4382-85c7-cd918c53cff8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.144713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.145426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.145704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.145981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.146621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:08.146924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.148352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.154867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.155633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.155988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.158985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.159893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:08.160359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.162018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.169821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 297/578] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:08.169821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.180182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 282/579] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:08.181414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.300688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063b4a5-7082-406d-af4e-b2a50f7861a6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.314326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96709ce1-f761-4b04-898e-86c5534cdec3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.318004 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.339806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.340437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.340713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.340984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.348662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.349284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.349562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.349831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.351408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:08.357560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:08.358518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:08.359388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 283/580] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:08.360699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.364148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.365618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:08.378512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:08.380114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:08.381297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:08.385734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 298/581] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:08.386231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.389101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.467987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4a48d-2c89-411c-a4e4-19d3d397e2dc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.496125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1df023-3543-41e3-8600-e6355ce0ae42 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.507024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.507671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.508308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.508596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.515786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.517209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:08.529070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:08.529967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:08.530862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 284/582] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:08.532011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.536631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.537333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.537575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.537874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] PATH_INFO: `/v3/users/glance` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.546836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.548300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:08.559689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:08.560620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 299/583] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:08.561601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.571821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.635676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.645936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de9134-d3b6-40cd-9176-344c449d36a6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.675995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d681e-8442-479c-a1a6-0fdc164e26fe None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.685878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.686491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.686767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.687039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.687727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:08.688034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.689401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.698324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 285/584] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:08.699098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.717895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.718870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.719636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.720092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.720993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:08.721486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.723794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.747120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 300/585] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:08.747120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.804700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.810752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798f91f-ba42-4ff1-bb83-b1b7b1de350f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.847803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.848413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.848694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.848966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.849546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7406177f-6a7b-4863-8508-3685faf04001 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:08.849649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:08.849943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.851370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.864887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 286/586] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:08.865600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.876524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:08.891709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:08.892325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:08.892603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:08.892872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:08.893595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:08.893900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:08.895308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:08.920104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 301/587] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:08.920104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:08.981310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5bb456-9251-4d50-956a-72a703d1df94 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.015536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.016201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.016484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.016765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.025790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9277aa-3dec-42b2-93e2-f0c8aadc121c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.027500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.029808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:09.049638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:09.050665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:09.051618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:09.051618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:09.051618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:09.051618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 287/588] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:09.051618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.065165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.071796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.073011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.073300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.073732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.084498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.085957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:09.101032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:09.102036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:09.103034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:09.103034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:09.103034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:09.103034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 302/589] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:09.103034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.135358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.168936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec9102-b75f-4b70-b16c-f328d0d68595 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.206751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.207404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.207715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.207988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/8a9090ab0ffa4b7eafcc742880237229/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.216266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7bd39-5efe-42b4-a054-4eaa74a59f05 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.223137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=8a9090ab0ffa4b7eafcc742880237229, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.224733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.238380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 288/590] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:09 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/8a9090ab0ffa4b7eafcc742880237229/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:43:09.239443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.271485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.273192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.274179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.274630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.291714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.293641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:09.315566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:09.316802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:09.317957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:09.317957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:09.317957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:09.323580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 303/591] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:09.323580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.330187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.394603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39718bb-f2f1-417f-bfdc-b1c36d73eabf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.426815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.447788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.448410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.448687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.448965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.449617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:09.449928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.450037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a3e05-9dc5-4985-bd69-a0843d24d33d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.451309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.465426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 289/592] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:09.466677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.497570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.498188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.498475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.498752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.499724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:09.500040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.501615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.523636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 304/593] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:09.524835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.597224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05167fc2-d1e8-48df-846d-f9da24452d1b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.620845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.637231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.638205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.638691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.639200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.640194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:09.640743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.642339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.660975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52db3be-a4dd-424e-b5b9-c6b2e2bfb000 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.661550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 290/594] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:09.662598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.704064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.704708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.705049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.705329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.715391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.717154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:09.733141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:09.734131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 15:43:09.735115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:09.739026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 305/595] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:09.739923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.743752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.782716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111170a2-7c40-432b-b057-a4151029ab17 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.821225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.822287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.822824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.823300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.830607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.832402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:09.845815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:09.846757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:09.847757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:09.850250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 291/596] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:09.851241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.877694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c531e4c2-35b4-4105-8ee3-9f8b3ec647df None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:09.908061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:09.913901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:09.914800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:09.915257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:09.915764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:09.923651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:09.925302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:09.936810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:09.937691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:09.938676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:09.940441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 306/597] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:09.941335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:09.973584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d46e8b-ea3c-450d-afff-5f06e6cd9169 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.009268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.012645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.013261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.013541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.013818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.014581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:10.014891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.016337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b80317-a960-423e-8af0-04cd018c851c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.034240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 292/598] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:10.035634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402c08c-7ee1-4df9-870e-d6997af1792e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.035894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402c08c-7ee1-4df9-870e-d6997af1792e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.036166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4402c08c-7ee1-4df9-870e-d6997af1792e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.039650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 293/599] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:10.039650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402c08c-7ee1-4df9-870e-d6997af1792e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.060611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8a06d3-9f9f-47a4-a852-17598589efcf None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.097597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.098275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.098504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.098811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.099506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:10.099840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.101267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.119269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 307/600] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:10.119269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.152599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.160570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4402c08c-7ee1-4df9-870e-d6997af1792e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.200626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.201243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.201528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.201804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.202520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:10.202809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.204281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995d3ef8-aa08-44b2-8b1f-b4d19a0bfcd0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.216617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 294/601] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:10.221220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.221497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.222006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.241655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:10.243007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:10.251345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2628f398-62e7-4743-93e6-b943ad470032 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.274800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.287814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.288441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.288720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.288991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.299879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.301362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:10.317746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:10.318875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:10.319835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:10.319835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:10.319835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:10.319835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 308/602] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:10.319835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.325432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 295/603] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:10.326459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.432738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce65eb5-42e8-4a52-8807-252c3aac4955 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.436525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.451911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c68e37-2e4c-4a5c-ae15-bd6840bd1442 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.477896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.479001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.479537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.480091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/396fc6e85a65437898bc9d964e429de3/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.489964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.490703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.491054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.491398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/257da2327d944095b51df9f677b3a99c/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.498326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=396fc6e85a65437898bc9d964e429de3, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.500074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.508796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=257da2327d944095b51df9f677b3a99c, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.510971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.515119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 309/604] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:10 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/396fc6e85a65437898bc9d964e429de3/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:43:10.516308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.541213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 296/605] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:10 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/257da2327d944095b51df9f677b3a99c/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:43:10.568037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.610396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.725289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76147e9-f9ab-42ca-8ed0-c86f5fe9a31c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.776435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.777116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.777584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.777690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.778297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:10.778600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.780032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:10.795590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 310/606] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:10.801163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.808462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d823e7-a898-479b-b737-678eabfebfd3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.857195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.858083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.858534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.858951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.866809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.871156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:10.876468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:10.877581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:10.878641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:10.881082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 297/607] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:10.889867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:10.910370 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:10.928861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6ba2a-383d-40d6-94ac-4f358dba2062 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:10.977692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:10.978622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:10.979096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:10.979531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:10.990606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:10.997733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:11.016932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:11.018069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:11.019217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:11.022492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3198f6-6e5e-49c1-9906-d0893020bdbe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:11.028925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 311/608] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:11.034431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:11.059972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:11.060584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.060864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.061445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.061800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:11.062104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:11.063531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.073151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 298/609] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:11.078673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:11.196989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:11.223374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbdf07f-2b1f-4c7a-a5f1-e57c1c1e7f0d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:11.258844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:11.259826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.260289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.260738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.261609 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:11.262118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:11.263777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.278629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47442a-db84-4613-9fec-e18c0ee4c82a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:11.294626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 312/610] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:11.298107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:11.303869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:11.343433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:11.344198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.344548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.344891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.357911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:11.359629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:11.389696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:11.390929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:11.392100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:11.392100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:11.392100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:11.394229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 299/611] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:11.399077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:11.497951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4b25c-b96f-499b-ba08-15ed0c666b65 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:11.544632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6063eef-be55-4d1f-9ef7-7bf20e54e450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:11.545594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6063eef-be55-4d1f-9ef7-7bf20e54e450 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.546042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6063eef-be55-4d1f-9ef7-7bf20e54e450 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.546463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6063eef-be55-4d1f-9ef7-7bf20e54e450 None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/690506f288f84debaadb4c8975759dad/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.566510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6063eef-be55-4d1f-9ef7-7bf20e54e450 None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=690506f288f84debaadb4c8975759dad, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:11.568323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6063eef-be55-4d1f-9ef7-7bf20e54e450 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.578454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38fc67-f264-4660-8f0e-b6ca23a71f33 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:11.591232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 313/612] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:11 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/690506f288f84debaadb4c8975759dad/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:43:11.592948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83c03d-ab68-4a85-b7a7-fa290f6f4877 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.593385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83c03d-ab68-4a85-b7a7-fa290f6f4877 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.593808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83c03d-ab68-4a85-b7a7-fa290f6f4877 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.595757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 314/613] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:11.596261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:11.607589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.607589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.607589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.626548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:11.627357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.627762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.628136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.628930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:11.629533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:11.631184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.638353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:11.639711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:11.647428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 300/614] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:11.656649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:11.751012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:11.785649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 315/615] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:43:11.833377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb441e6-d7a7-40fe-b9f8-84658c2acf75 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:11.886199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:11.886840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:11.887123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:11.887399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:11.899745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:11.901737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:11.918575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:11.919599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:11.920563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 301/616] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:11.925534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.029163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c064db6d-ad8f-434a-a598-c0f729d79aa4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:12.080223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:12.080846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:12.081127 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:12.081400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:12.082132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:12.082437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:12.083954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:12.102559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 316/617] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 15:43:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:12.108508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.172051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:12.251914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.275573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea35f36-406c-4b2c-bd07-36e89b5c529b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:12.324099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:12.334868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:12.335446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:12.335749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:12.336022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:12.358053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:12.359507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:12.387119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:12.388185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:12.389084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:12.389084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:12.389084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:12.389084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 302/618] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:12.393756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.416702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c36b1-3664-4805-8ac7-7d299ddbc47e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:12.476615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:12.477256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:12.477530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:12.477795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:12.498812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:12.503678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:12.504769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:12.505634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:12.507327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 317/619] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:12.512727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.555890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:12.565839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f74334-78d2-4b17-b7c7-b5fe5ffe1ed1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:12.639158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:12.639968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:12.640254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:12.640522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:12.641166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:12.641466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:12.642822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:12.663039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 303/620] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:12.671993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.729570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb93759-c41b-4a78-b9cb-5066b60d2ed7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:12.731439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:12.781043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da8be19-cf81-4806-b341-7408e59af5e2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:12.817030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:12.817668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:12.817988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:12.818266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:12.825548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:12.826984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:12.839881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:12.840799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:12.841721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 304/621] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:12.846776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.856100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:12.868262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:12.869399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:12.869695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:12.869971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:12.870695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:12.870952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:12.872388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:12.902550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 318/622] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:12.909161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:12.961658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e060565-8863-4a89-a5c0-18730baa92d4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.007090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.007733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.008059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.008600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.010881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:13.011738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:13.014228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:13.027518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 305/623] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:13.033422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:13.101212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:13.133358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab18128-242a-467b-b28e-78f5e948981c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.144324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a933f2-ffdb-4bb0-b0ba-f2d49e065652 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.175528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.176546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.176894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.177239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.185752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a78d16-2436-4e4a-a610-12982f501040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.186752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78d16-2436-4e4a-a610-12982f501040 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.187229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78d16-2436-4e4a-a610-12982f501040 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.187733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a78d16-2436-4e4a-a610-12982f501040 None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/f92a1a83dc5a4ababdebb85429de82b0/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.193763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:13.195387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:13.207228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78d16-2436-4e4a-a610-12982f501040 None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=f92a1a83dc5a4ababdebb85429de82b0, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:13.209899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a78d16-2436-4e4a-a610-12982f501040 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d09b513a-db4e-416f-bf81-60854928e1c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:13.212740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:13.213829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:13.214779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:13.214779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:13.214779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:13.214779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 319/624] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:13.219799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a325ce-282c-4f7f-b89d-b3162bac0363 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.219799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a325ce-282c-4f7f-b89d-b3162bac0363 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.219799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a325ce-282c-4f7f-b89d-b3162bac0363 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.221523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 320/625] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:13.222531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a325ce-282c-4f7f-b89d-b3162bac0363 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:13.235177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 306/626] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:13 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/f92a1a83dc5a4ababdebb85429de82b0/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:43:13.242671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a123f-264d-41a5-8cc4-6ebeda279713 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.243286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a123f-264d-41a5-8cc4-6ebeda279713 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.243772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a123f-264d-41a5-8cc4-6ebeda279713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.245368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:13.284883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ca7a123f-264d-41a5-8cc4-6ebeda279713 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:13.286120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ca7a123f-264d-41a5-8cc4-6ebeda279713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:13.366635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a325ce-282c-4f7f-b89d-b3162bac0363 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.373250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 307/627] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:13.374608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09637-e31b-4043-b4c0-60fef0427192 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.374899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09637-e31b-4043-b4c0-60fef0427192 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.375172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f09637-e31b-4043-b4c0-60fef0427192 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.380772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 308/628] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:13.382108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.382389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.382659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.402225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:13.403446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:13.435691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.435691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.436001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.436220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.437386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:13.437386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:13.438577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:13.453382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:13.454458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 321/629] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:13.461244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:13.480922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 309/630] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:13.541608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:13.565494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c290-e7b6-4f35-acb3-01a76a8fea0a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.599912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.600552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.600835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.601110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.609118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:13.610575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:13.622366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:13.623334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:13.624334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 322/631] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:43:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:13.628936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:13.723928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:13.765955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a4b6d-307e-4721-aec6-9e6b417b75f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.802246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.803225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.803732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.804205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.805139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:13.805650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:13.807353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:13.827120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 310/632] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:43:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:13.832773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:13.865353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:13.886516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439b91e-7102-4b78-96da-dffacba2ebc6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.930986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:13.932272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:13.932551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:13.932820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:13.936046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:13.949449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:13.951280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eac8052-3247-4f81-8e04-984631f98e38 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:13.955425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:13.956381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:13.957241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:13.959932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 323/633] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:13.960987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.000251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce1a098-9da6-4426-804e-8aa5eb303c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.001377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a098-9da6-4426-804e-8aa5eb303c95 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.002158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a098-9da6-4426-804e-8aa5eb303c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.002897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce1a098-9da6-4426-804e-8aa5eb303c95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.093949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1a098-9da6-4426-804e-8aa5eb303c95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.096524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce1a098-9da6-4426-804e-8aa5eb303c95 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.097925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90dffdf-3574-49c3-ae9b-43714ef4756c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.136303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.139106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.139789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.140072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.140346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.153012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:14.159213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:14.160349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:14.161199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 324/634] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:14.161908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.218376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 311/635] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:14.220218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5bcde-838c-4917-8b7c-acd9e2f863d3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.220505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5bcde-838c-4917-8b7c-acd9e2f863d3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.220789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5bcde-838c-4917-8b7c-acd9e2f863d3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.227504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 312/636] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:14.229020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5bcde-838c-4917-8b7c-acd9e2f863d3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.232419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.280992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00632035-0b5a-421f-91c9-0aaf4ba4df9a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.317611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.318224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.318498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.318780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.319446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:14.319788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.321172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.333527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 325/637] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:14.333527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.343573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5bcde-838c-4917-8b7c-acd9e2f863d3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.374294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.379723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.380537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.380821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.381097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.381761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:14.382128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.383496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d7831-c189-4b95-848d-4c18d6c8755f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.391929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 313/638] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:14.393071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.393358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.393634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.411187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:14.412415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:14.444686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff454241-5de6-4b44-a69a-47eb257b2702 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.462333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.487567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 314/639] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:14.487567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.488981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.489510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.489854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.490227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.502336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.504071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:14.519677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:14.520866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:14.522079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:14.522079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:14.522079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:14.527680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 326/640] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:14.527680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.618424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10344210-c5cd-47a3-934d-662398f295c5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.633688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d439d4b6-1ae1-4b50-ac6c-b2feac299c05 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.655942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.664563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.665482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.666277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.666553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.678836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.679027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.679711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.680086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.680466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.680573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.693667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.695240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:14.699461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:14.700611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:14.701737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:14.701737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:14.701737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:14.701737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 315/641] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:14.701737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:14.715471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:14.716675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:14.717740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:14.717740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:14.717740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:14.723903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 327/642] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:14.724571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.756946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.827129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d2c70-b595-4e7c-acf2-a79afb20137b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.845101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa15d4-7316-4be2-9380-f58644d6ebd5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:14.863936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.864544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.864821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.865092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.865725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:14.866031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.867380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.878911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 316/643] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:14.879660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.885037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:14.885833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:14.886110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:14.886379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:14.887020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:14.887321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:14.888734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:14.901714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 328/644] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:14.906591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:14.916241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.991790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:14.996884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684152ea-e7bc-4e17-9fee-6bf8eb5a07d0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.012751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f8c6b-5b68-4c9f-8c73-63e8b0f5a7c6 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.034219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.034835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.035116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.035389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.036061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:15.036363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.037744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.053841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 317/645] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:15.053841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.055017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.055694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.055977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.056249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.067637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:15.072557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:15.073514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 329/646] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:15.074496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.161396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.168128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99fdd4-6460-4215-b58d-6b2c834f04dc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.187965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47cf0fb-338b-4c32-bda5-346012227f3f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.203198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.203863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.204154 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.204413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.211082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.213085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.224419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:15.224597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:15.225454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:15.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:15.226794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.226794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.226930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 318/647] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:15.226930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.230393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.237972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.239435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:15.251422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:15.252457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 330/648] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:15.258050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.259773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.342285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068b35a-06e1-489d-af30-4c80b78aba8b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.363092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dabea5-3f67-4508-a852-aa081a83dc9f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.377672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.378369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.378791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.379122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.387484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.389251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.399252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.399901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.400226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.400501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:15.400796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:15.401701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 07 15:43:15.408204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:15.408768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:15.408768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.408906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 319/649] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:15.409010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.409098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.414586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 331/650] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:15.417559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.428231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.491879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.523373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b764694-2ea3-4905-96bf-899c6371e733 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.536471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ab7ea-3a45-4dc3-a4a5-3998842c64a1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.571998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.572610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.572892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.573161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.574462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.575139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.575487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.575851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.576642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:15.577020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.577673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:15.577992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.578652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.579408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.592217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 320/651] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:15.593039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.597051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 332/652] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:15.597852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.672336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.694649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b33f915-872b-4e62-82f2-e06085d94515 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.695129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ef39a-d827-4e2c-954c-0537c4faf27d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.717393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.727829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.728514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.728794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.729069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.729644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.730041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:15.730351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.730673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.731012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.731343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.731787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3352e8cd-4b30-4c1c-9870-3d154f8b6efc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.741485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.742898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.748904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 321/653] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:15.750158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f93d2-eb72-4d98-ba84-5d484e53651a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.750449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f93d2-eb72-4d98-ba84-5d484e53651a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.750718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f93d2-eb72-4d98-ba84-5d484e53651a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.751949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 322/654] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:15.752637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f93d2-eb72-4d98-ba84-5d484e53651a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:15.762398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:15.763470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:15.764479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:15.764479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:15.764479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:15.764479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 333/655] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:15.764479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.852727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f93d2-eb72-4d98-ba84-5d484e53651a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.862662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1698e8a-f8c7-4ddf-84ba-d13119f40b85 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:15.890251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.890879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.891368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.891873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.892639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=8a9090ab0ffa4b7eafcc742880237229) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:15.892947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.894589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:15.896327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e0c0e-8dfd-4864-8b2e-5f96172e050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:15.896936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0c0e-8dfd-4864-8b2e-5f96172e050c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:15.897212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0c0e-8dfd-4864-8b2e-5f96172e050c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:15.897480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e0c0e-8dfd-4864-8b2e-5f96172e050c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:15.897616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.903581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 323/656] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/role_assignments?user.id=8a9090ab0ffa4b7eafcc742880237229&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:15.904372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:15.955853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:15.985185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e0c0e-8dfd-4864-8b2e-5f96172e050c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:15.987186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e0c0e-8dfd-4864-8b2e-5f96172e050c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.024484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb8f5e-6405-4934-9f39-88ca5c33cb76 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.063825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.064447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.064731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.065001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.100573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 334/657] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:16.101969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.102258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.102531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.120923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:16.122109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:16.145580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.147945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.149453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.201444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 335/658] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:16.202360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.208685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.270140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 324/659] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:16.275478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.331702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c76f709-7d7e-42bd-b1ea-942252ad8356 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.369971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.370978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.371522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.372005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.372828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:16.373328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.375102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.377001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.380964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e18c1-9f05-40cb-a62e-17e6a9ebef29 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.388933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 336/660] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:16.390313 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.418176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.418981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.419226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.419584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.429939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.431585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:16.446786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:16.447947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:16.449110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:16.449110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:16.449110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:16.450980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 325/661] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:16.451985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.464670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.511076 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17390a33-cf61-436f-94be-b0ce55806e47 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.546203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.546840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.547120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.547391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.559040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.560346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745baa-742d-4e1f-bb5a-47d17f6afe1b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.560525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:16.576383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:16.577565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:16.578616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:16.578616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:16.578616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:16.578616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 337/662] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:16.578616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.602964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.605025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.605613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.606139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.616301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.618660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:16.630507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:16.631483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 326/663] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:16.633133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.649971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.687644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1e759-97c8-4df2-8cdc-b23fef089f20 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.707473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.730358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.730903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.731186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.731467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.732164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:16.732472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.733864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.744361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8a218c-eb64-4a95-aad7-92ff4962c844 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.746444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 338/664] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:16.747294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.786667 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.787640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.788102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.788555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.800316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:16.805611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:16.806654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:16.807525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 327/665] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:16.808790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.857327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825103e8-3740-4e9f-92c8-fad2706b8a38 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.879267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.895168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.895832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.896082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.896357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.897074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:16.897312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.898680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.910636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 339/666] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:16.911411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:16.919406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ac7656-1fae-4899-af5e-6b9a6cbc9462 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:16.946840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:16.957939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:16.958678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:16.958961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:16.959232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:16.959982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:16.960286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:16.961708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:16.987663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 328/667] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:16.987663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.021692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaba91f-227c-4266-b7dd-0502ff31274b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.059686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.060459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.060694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.060966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.067894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.069336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:17.081728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:17.082663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 340/668] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:17.083668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.112683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fd760-e419-4e17-997e-604108a0185a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.143415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.154423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.155212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.155672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.156096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.157319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:17.157791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.159595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.173249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 329/669] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:17.174778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.192948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.199246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d2816-194c-40ed-86d1-9de114941bbd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.236674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.237312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.237590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.237911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.245109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.246556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:17.258012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:17.258912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:17.260440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:17.264284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 341/670] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:17.265094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.301396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cf909-0bfd-4fbf-97fa-44ec5d7598c3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.340173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.340782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.341066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.341338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.375667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32257ae8-6aa8-427c-ae21-1b1b7729f998 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.409822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.414362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.414939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.415221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.415492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.416244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:17.416550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.417990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.421916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.423363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.430417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 342/671] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:17.432591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.455564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.528038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 330/672] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:17.529237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.549103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53456d-80af-4ff8-a79a-0ae76f6f1d74 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.585857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.586494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.586774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.587044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.600344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.602107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:17.621339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:17.622352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:17.628298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:17.628298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:17.628298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:17.628298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 343/673] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:17.628298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.632059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.644887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c6051-0352-4747-9836-764d3b782ee1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.682267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.682879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.683168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.683443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.684195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:17.684531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.685957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.697864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 331/674] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:17.698895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.703440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.739799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf70d55-ec3b-4ed8-91f6-3c9a9239f782 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.776777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.777447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.777728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.778188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.778865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=396fc6e85a65437898bc9d964e429de3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:17.779173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.780644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.795569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 344/675] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/role_assignments?user.id=396fc6e85a65437898bc9d964e429de3&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:17.795569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.831261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b832b3-e69e-4308-a4d7-cce163009985 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.885709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.886539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.886862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.887289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.902608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.903716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:17.919773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:17.921662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:17.923240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:17.923240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:17.923240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:17.924482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6551c-3434-4b1f-82c0-b42253e03d77 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:17.927257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 332/676] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:17.929047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:17.933180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.975023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:17.981947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:17.982922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:17.982922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:17.983123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:17.985562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:17.985562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:17.985742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.003025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 345/677] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:18.003889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.070814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f1cae-f1b5-4115-9713-db90cea299ff None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.109913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.110877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.111334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.111826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.112801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=257da2327d944095b51df9f677b3a99c) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:18.113289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.114886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.121721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 333/678] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:17 2022] GET /identity/v3/role_assignments?user.id=257da2327d944095b51df9f677b3a99c&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:18.122832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.154255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94104d9b-7ea7-4036-9eee-dcd03aff09c5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.204595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.205228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.205509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.205785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.206553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:18.206867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.208517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274facf9-a950-427d-9de3-f9cacf6423c1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.221261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:18.223708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 346/679] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:18.225208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4c8d-17be-4693-95ca-8b14c2ebbb22 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.225649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4c8d-17be-4693-95ca-8b14c2ebbb22 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.226100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182e4c8d-17be-4693-95ca-8b14c2ebbb22 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.228527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 347/680] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:18.237057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e4c8d-17be-4693-95ca-8b14c2ebbb22 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.269944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9968ab24-043e-4543-af75-cea0b2b18402 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.307271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:18.317210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.317866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.318142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.318413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.331585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.334152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a7f2cd04-2b6d-4e3f-a35b-e3a553d09243 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:18.347685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:18.348872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:18.350074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 334/681] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:18.350826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.350907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.351195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.370747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:18.371953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:18.374915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182e4c8d-17be-4693-95ca-8b14c2ebbb22 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.413049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.414352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.415014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.416111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.424481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.427400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:18.447531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:18.448757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:18.449874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:18.450648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 348/682] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:18.451307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.461005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 335/683] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:18.462344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.523633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:18.616036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe7176-7ba5-46b5-b5ef-73d49aff4d9b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.618095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a616959-2a21-4ef0-9e0b-3b079c575a4c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.643092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:18.670260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.670871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.671145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.671723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.672701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:18.673008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.674911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.691658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 336/684] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:18.694436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.697187 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.697926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.698435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.698802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.699676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:18.700083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.701669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.737799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 349/685] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:18.743242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.828309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40e531-854c-4df4-a911-ed9924cdd270 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.846377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:18.866775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.867309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.867609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.867885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.868608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=690506f288f84debaadb4c8975759dad) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:18.869463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.870892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:18.874510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b824a-424f-4118-826b-c52c047f5428 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:18.877392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 337/686] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/role_assignments?user.id=690506f288f84debaadb4c8975759dad&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:18.878240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:18.911917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:18.912697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:18.913078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:18.913516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:18.959908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:18.987727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:18.989379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.034019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef21f6c-6a1b-49af-ad2c-cb05b9e7fe52 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.096244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 350/687] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 07 15:43:19.104968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.106898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.107575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.107852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.108121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.121818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:19.127094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:19.129344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:19.131378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 338/688] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:19.135844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.190744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:19.255536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c30a353-d196-43cd-8ba0-a1235484cfec None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.259017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:19.283890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32571cff-eab9-43e8-bcac-961607499b9d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.296718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.297602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.297887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.298160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.298825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:19.299125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:19.301107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.311572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 339/689] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:19.317897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.342781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.343433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.343742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.344015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.376892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:19.378313 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:19.417107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:19.419361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:19.421456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:19.421456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:19.421456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:19.427871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 351/690] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:19.428739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.442165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006b1fa-d94d-4475-88fe-ae29b9bc94d2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.443714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:19.479157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.480652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.480930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.481205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.491868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:19.493615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:19.511568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:19.513873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:19.516062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:19.516062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:19.516062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:19.516062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 340/691] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:19.519564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.628789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957097b-c6d1-4245-8476-8dd2b1cd7efc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.668994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.669608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.669915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.670415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.671618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:19.671926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:19.673302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.687956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 341/692] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:19.697161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.714562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a620c4e-b02b-4c83-87b7-fc0e4151fc93 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.730934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:19.754121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:19.771173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.771867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.772154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.772429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.773082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:19.773391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:19.774779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.805163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 352/693] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:19.811218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.852005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a9a63-81db-4258-84dd-d9884b2d0ca3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:19.888923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:19.889661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:19.890044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:19.890418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:19.898119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:19.899683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:19.916491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:19.918710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:19.921033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 342/694] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:19.929377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:19.980512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cad7114-41c1-4bd2-b0b6-a79f59124a55 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:20.039989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:20.054261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5d10-9573-4e95-8c10-75a9beef3d00 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:20.091623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:20.092400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.092686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.092959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.093689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:20.093997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:20.095483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:20.116018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 343/695] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:20.116349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:20.117023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.117307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.117585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.127837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:20.139537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:20.140969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d08c8da1-4a12-450c-bc29-311ade6fd7d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:20.160081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:20.161027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:20.162011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:20.164463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 353/696] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:20.165921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41e543-d98f-4f92-946c-f2a22366f0d8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.166354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41e543-d98f-4f92-946c-f2a22366f0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.166772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb41e543-d98f-4f92-946c-f2a22366f0d8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.168937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 354/697] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:20.169851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41e543-d98f-4f92-946c-f2a22366f0d8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:20.230787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:20.271452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bee9715-2a8e-4ff3-8143-5fe2d5bb1525 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:20.309583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:20.310732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb41e543-d98f-4f92-946c-f2a22366f0d8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:20.316831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:20.317565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.317994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.318370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.355809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:20.356902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.357352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.358161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.359279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:20.359763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:20.361626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea8cd0a-aa26-42e1-ae15-df50e4c56dcb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:20.378201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 355/698] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:20.379707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.380120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.380518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.404125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:20.406143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:20.407285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:20.409152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:20.491576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 356/699] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:20.491576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:20.544481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:20.574424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 344/700] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:20.579948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:20.631358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:20.671453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b67c3-fd3d-4a1b-8dd9-fe01830834ca None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:20.734677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da02ed-c57f-4d34-a59c-613d36b1b83a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:20.761720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:20.762374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.762626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.762904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.763902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=f92a1a83dc5a4ababdebb85429de82b0) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:20.764023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:20.765391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:20.781485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 357/701] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:20 2022] GET /identity/v3/role_assignments?user.id=f92a1a83dc5a4ababdebb85429de82b0&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:20.782705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:20.783376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:20.783686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:20.783967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:20.802573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:20.804152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:20.829056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:20.830078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:20.831694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:20.831694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:20.831694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:20.831694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 345/702] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:20.835897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:20.924806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:20.986563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:20.993019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229915d9-bcce-44da-864d-e8885e346778 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.040261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.040496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.040665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.040885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.041622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:21.041911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.043185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf0899b-b922-4c1e-b315-7a0d968f1298 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.043471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:21.060588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 358/703] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:21.062432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0aa65-c560-486d-bacb-8291de130121 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.062432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0aa65-c560-486d-bacb-8291de130121 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.062432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb0aa65-c560-486d-bacb-8291de130121 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.066044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 359/704] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:21.066044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0aa65-c560-486d-bacb-8291de130121 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:21.069491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab7f5a-1059-4520-82e3-6d2b20c1e485 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.111228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.112021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.112300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.112573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.113158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.114976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-2bb57dc8-0646-491c-9502-7423f38ca64c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:21.129813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:21.131149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:21.132300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 346/705] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Sep 7 15:43:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:21.133344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.133438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.133755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.159777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:21.161005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:21.189705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb0aa65-c560-486d-bacb-8291de130121 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.225537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.226514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.227015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.227512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.235137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.236363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 347/706] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:21.237292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.237404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:21.250102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:21.251081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:21.252020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:21.254447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 360/707] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:21.260045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:21.300008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:21.380091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:21.416913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a88f82-a50c-4a3e-a539-34311161929d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.459493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5a0e2d-ed9f-4d82-be61-88e7de810d53 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.481866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.482494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.482774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.483047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.483708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:21.484012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.485371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.496012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 348/708] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:21.500840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:21.519716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.520936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.521501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.522171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.523389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:21.523929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.526119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de5d32-1e29-4c31-ad25-3a5443a063ea None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.544276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 361/709] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:21.546326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bc961-03d2-4281-815b-d8f38f2496b2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.546830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bc961-03d2-4281-815b-d8f38f2496b2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.547671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bc961-03d2-4281-815b-d8f38f2496b2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.553677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 362/710] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:21.553677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bc961-03d2-4281-815b-d8f38f2496b2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:21.613808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd1199-90a6-4eaa-84ec-4f088d6c2dec None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.635857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:21.651510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.652247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.652603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.653003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.653661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:21.654038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.655484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42672879-ffac-4392-b308-a7f43a3eb1f1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.661362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 349/711] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:21.661810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bc961-03d2-4281-815b-d8f38f2496b2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.683584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940599-396d-4117-b8fd-32051485f474 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.683584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940599-396d-4117-b8fd-32051485f474 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.683584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce940599-396d-4117-b8fd-32051485f474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.696201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:21.702464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ce940599-396d-4117-b8fd-32051485f474 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:21.703728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ce940599-396d-4117-b8fd-32051485f474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:21.715126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.716219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.716621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.717314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.718447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=8a9090ab0ffa4b7eafcc742880237229) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:21.718857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.721055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.729638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 363/712] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/role_assignments?user.id=8a9090ab0ffa4b7eafcc742880237229&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:21.730987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:21.784381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 350/713] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:21.866423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f102049b-dae0-4a1b-9b7d-38fee80ebb07 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:21.915147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:21.915915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:21.916193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:21.916462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:21.927063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:21.930170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:21.932254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:21.949851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:21.950969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:21.951938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:21.951938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:21.951938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:21.954775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 364/714] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:21.958489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce940599-396d-4117-b8fd-32051485f474 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:22.062877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce940599-396d-4117-b8fd-32051485f474 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:22.090637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:22.112921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:22.113900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:22.114408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:22.115071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:22.117449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:22.117968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:22.119639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:22.124394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:22.140957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 351/715] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:22.146384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:22.281774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862607c3-292c-460b-81bb-3c5ed1404b53 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:22.319490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd714-8caa-472d-b180-14fb11b03cd3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:22.363061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:22.364090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:22.365037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:22.365577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:22.366475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:22.368476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:22.374008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:22.374909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:22.375388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:22.375885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:22.376687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:22.378352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:22.400086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:22.401192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:22.402064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:22.405226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 365/716] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:22 2022] GET /identity/v3/services/network => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:22.409618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 352/717] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:43:22 2022] POST /identity/v3/users => generated 254 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:43:22.411708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:22.481809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:22.530117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7b3323-75de-46e9-96a2-28696921b7f5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:22.567686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:22.568316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:22.568594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:22.568865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:22.569560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] RBAC: Adding query filter params (name=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:22.569807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:22.571210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c392-6993-41f5-8f3e-8bdc61911820 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:22.577035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 366/718] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:22.582484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:22.677322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:22.734317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cafca9-a2c8-4c1c-a8f1-e8dd0a59bde6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:22.771926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:22.772551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:22.772833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:22.773109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:22.773758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] RBAC: Adding query filter params (type=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:22.774061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:22.775503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e35aa5-bbb0-4a84-bf28-80ac03bb8c00 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:22.780796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 353/719] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:23.364467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8561a-aa35-4ed2-8920-3b93f1a6c133 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:23.365464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8561a-aa35-4ed2-8920-3b93f1a6c133 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:23.365948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8561a-aa35-4ed2-8920-3b93f1a6c133 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:23.367771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 367/720] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:23.373264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:23.373699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:23.373786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:23.401115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:23.402325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:23.482503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 354/721] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:23.887339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8561a-aa35-4ed2-8920-3b93f1a6c133 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:24.003198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8561a-aa35-4ed2-8920-3b93f1a6c133 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:24.030694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:24.047014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:24.047723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.048007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.048340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.063433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:24.068292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:24.069267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:24.070120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:24.073887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 368/722] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:24.080636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:24.133561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:24.218308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80467c7f-4c12-4af7-b6ed-b526924f1e7b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:24.262960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:24.263585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.263870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.264141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.264805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:24.265109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:24.266472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f8c788-f907-40e7-a23c-8a7144be9a0c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:24.275401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 355/723] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:24.281107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:24.320895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ebf3-c773-428e-aa68-74da7993f88b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.321218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ebf3-c773-428e-aa68-74da7993f88b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.321443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509ebf3-c773-428e-aa68-74da7993f88b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.323363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 356/724] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:24.349381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21515ec-68a6-4833-a75f-0a28b881bf23 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.349657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21515ec-68a6-4833-a75f-0a28b881bf23 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.349937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21515ec-68a6-4833-a75f-0a28b881bf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.384029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f21515ec-68a6-4833-a75f-0a28b881bf23 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:24.386116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f21515ec-68a6-4833-a75f-0a28b881bf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:24.413093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283484cd-f74e-42b4-a178-38c9fc6bdc9c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:24.456695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:24.458605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.460068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.460543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.483734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:24.485611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:24.510445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:24.511571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:24.512525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:24.512525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:24.512525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:24.514708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 369/725] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:24.520335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:24.550856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 357/726] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:43:24.552407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73065483-fb3c-45fb-a5d6-797422658ce9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.552695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73065483-fb3c-45fb-a5d6-797422658ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.552970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73065483-fb3c-45fb-a5d6-797422658ce9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.554471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 358/727] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:24.559500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.559821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.560095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.572483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:24.579810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:24.581003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:24.617369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:24.654895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 359/728] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 07 15:43:24.722460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7202da7-c034-475f-a3d8-06fa685b394f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:24.771012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:24.771663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.772006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.772286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.772930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:24.773237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:24.774613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:24.792962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 370/729] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:24.798192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:24.897661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:24.909075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eed63-43a7-4c31-b6e6-50fa783c73cb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:24.947109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:24.947890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:24.948176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:24.948451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:24.956116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:24.957660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:24.970833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:24.971831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:24.973482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 360/730] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:24.977429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.008021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.062224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.072808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4b4675-5caa-4f6b-a18f-6086bd9e8f60 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.093891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462bc233-2507-4a0f-9836-0cfb012c5036 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.112706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.113472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.113700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.113982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.114928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:25.116158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:25.143570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 371/731] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:43:24 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:25.143570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.166286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.167120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.167571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.168003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.169009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:25.169464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:25.171088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141fb44-39a0-488d-9b1f-71ae33059bab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:25.201914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 361/732] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 15:43:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:25.203294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17cb20-48f9-4b63-90cd-4550666f9a1a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.203600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17cb20-48f9-4b63-90cd-4550666f9a1a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.203876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17cb20-48f9-4b63-90cd-4550666f9a1a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.209472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 362/733] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:25.210398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17cb20-48f9-4b63-90cd-4550666f9a1a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.265503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c0fc52-ff4f-47f4-b8b6-bbb89dfc0df0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.304000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.305254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ce6714-3b6c-455a-b722-5174e673894c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.305933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6714-3b6c-455a-b722-5174e673894c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.306361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6714-3b6c-455a-b722-5174e673894c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.306708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ce6714-3b6c-455a-b722-5174e673894c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.320189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ce6714-3b6c-455a-b722-5174e673894c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d4ce6714-3b6c-455a-b722-5174e673894c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:25.326974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:25.328283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:25.329384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:25.330197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17cb20-48f9-4b63-90cd-4550666f9a1a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.330295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 372/734] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:25.330295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.330295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.330295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.350002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:25.351323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:25.361208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.371321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.372286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.372630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.373051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.433319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 373/735] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:25.434877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.454893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:25.456842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:25.596876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.625865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca46fff-6ad1-4ce5-bc34-4658f3eec6cf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.628745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 363/736] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:25 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 07 15:43:25.637056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.690796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.691413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.691742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.692017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.692684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:25.693124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:25.694758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3059f3e2-2a81-4b60-86c8-bbc21e18905b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:25.707720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 374/737] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:25.709096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe158e-d020-4a2a-99f4-d4c8c70b2137 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.709386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe158e-d020-4a2a-99f4-d4c8c70b2137 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.709665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe158e-d020-4a2a-99f4-d4c8c70b2137 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.711005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 375/738] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:25.716956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe158e-d020-4a2a-99f4-d4c8c70b2137 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.723884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.770976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e3e555-e009-49d5-bc9b-4b494982325b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.808079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.808734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.809015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.809289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.819725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:25.821162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:25.835112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe158e-d020-4a2a-99f4-d4c8c70b2137 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-12557bb6-8a6f-4f10-a6c0-d1f245fa97fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:25.836426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:25.837660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:25.838622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:25.838622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:25.838622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:25.838622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 364/739] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:25.838858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.839426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.839426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.857751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:25.858881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:25.878118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:25.878752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:25.879028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:25.879297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:25.887529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:25.890917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:25.892400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:25.908848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:25.909846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:25.910750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:25.910750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:25.910750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:25.910750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 376/740] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:25.910750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.936072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 365/741] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 07 15:43:25.936897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:25.984333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.034951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d97e3f-9370-43b8-b1d0-2f5703b54041 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.059937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f287a03a-eaae-4322-a61f-072fe8dee27e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.099974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.100950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.101407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.101900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.102755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:26.103784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.105637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.124726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 366/742] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:26.126008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.159281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.191519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.192377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.192752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.193200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.194010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.195783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.219010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 377/743] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:43:25 2022] POST /identity/v3/services => generated 240 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:26.220109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.263403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962687d-19c8-4583-a2d8-da33f70d9d89 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.310784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.311765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.312225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.312674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.313655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:26.314230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.315865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.331284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 367/744] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:26.332629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.379714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f7ba-c72f-47af-b19a-6826fba5cf64 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.403197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.427816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.428454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.428736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.429039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.431631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.437937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.439391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:26.452830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:26.453786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 378/745] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:26.454687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.461954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994ab96-dc1f-45b2-a024-d9e022611b12 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.501833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.502820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.503324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.503824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.517853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:26.524424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:26.525409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:26.526276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 368/746] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:26.526989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.580511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc26e31-6760-4009-a740-3d4dc83220b2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.616035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.617038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.617520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.618014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.625678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.627365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:26.640103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:26.641171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:26.642131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:26.642692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02d3a8-2f62-4326-8df2-84fd1e65129b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.644752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 379/747] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:26.645736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.681359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.681965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.682242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.682510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.683647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:26.683647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.684166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.685010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.703496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.704390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 369/748] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:26.705449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.773485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87efab1-d0e5-46a7-bbe4-5846f813bc18 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.822163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.823407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.824341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.825078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.826188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:26.826718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.828965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.832887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4248f-4bcd-42ff-8978-c0da8eee26af None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:26.844539 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 380/749] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:26.844702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.880948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:26.881502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:26.881782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:26.882054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:26.882773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:26.883077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:26.884803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:26.903189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 370/750] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:26.904077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:26.968831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.977098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:26.981287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40e828-1606-4e3f-b055-9b8f5bb9c120 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.019017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.019637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.019896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.020164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.020889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=f92a1a83dc5a4ababdebb85429de82b0) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:27.021195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.022806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.026396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e37242-a684-494e-ab55-59fa39c1fb67 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.034911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 381/751] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/role_assignments?user.id=f92a1a83dc5a4ababdebb85429de82b0&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:27.036107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.072114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.073412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.073897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.074515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.087728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.089918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:27.109674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:27.111639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:27.111639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:27.111639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:27.114233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 371/752] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:27.121191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.250506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f94dc8-e240-48c9-987c-d8f5899e9311 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.270713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:27.293296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.293908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.294183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.294451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.295096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:27.295398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.296777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.299785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b79d41-9cf8-42b2-a248-ec8745a2c6fd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.320868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 372/753] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:27.329140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.373489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.374133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.374416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.374693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.389914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.391361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:27.419309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:27.420345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:27.421256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:27.421256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:27.421256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:27.421256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 382/754] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:27.428314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.457489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:27.522231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077316cb-f77e-427b-9241-4d581eaee08f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.605706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d62dc9a-fb5d-490a-b0e5-1cdd973b99af None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.608260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.608918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.609199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.609472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.618845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.620679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-38ee4361-1386-497a-bcaf-8e41fa74597f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:27.633028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:27.634061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:27.635036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:27.637740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 373/755] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:27.639275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f194a6-bb23-48b9-a6d2-9a41eec8f932 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.639788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f194a6-bb23-48b9-a6d2-9a41eec8f932 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.640264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f194a6-bb23-48b9-a6d2-9a41eec8f932 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.645963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.645963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.645963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.645963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.646327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:27.646327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.646661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 374/756] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:27.647196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be1f45e-5fca-43f0-9097-303f3f62a35b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.647922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f194a6-bb23-48b9-a6d2-9a41eec8f932 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.663334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 383/757] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:27.669240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.670195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.670321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.689640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:27.690562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:27.690884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:27.771654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 384/758] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:27.773211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.776562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:27.804893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f194a6-bb23-48b9-a6d2-9a41eec8f932 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.889460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb574ddc-74d7-4805-b086-f124ac595391 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:27.893714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.894343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.894675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.894953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.895723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:27.896033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.897475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.931527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:27.932798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:27.933290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:27.933755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:27.934610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 375/759] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:27.939760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:27.941944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:27.943897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:27.957907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:27.958841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:27.959776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:27.963175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 385/760] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:27.970357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.088109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.136170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.139747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff92c894-c59e-4db1-bf47-ef7eb8623f68 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.148199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a791c1-aca8-42e7-9392-2654d4257eea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.177555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.178171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.178449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.178718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.179412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:28.179737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.181379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.194565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 386/761] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:28.199620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.204053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.205041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.205528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.206005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.280667 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.282599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.314121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ce5c-3456-4f23-be3b-f40eeb5b19d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.342502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.355585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.356325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.356606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.356875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.357459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.358935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:28.370687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:28.371774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:28.372642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 387/762] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Sep 7 15:43:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:28.373479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.393137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.409006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 376/763] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:43:28.410294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.487537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5003a9-811c-40b3-bede-4ff22a072f42 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.519915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fa527-5a1e-40ce-b37c-080906f4e20f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.524475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.525689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.525967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.526239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/8a9090ab0ffa4b7eafcc742880237229/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.528340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=8a9090ab0ffa4b7eafcc742880237229, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.529584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.546355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 388/764] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:28 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/8a9090ab0ffa4b7eafcc742880237229/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:43:28.547595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.567587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.568296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.568510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.569028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.569596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:28.569908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.571324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30888fe-17c6-4a88-b52c-f9ee9ac22273 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.582369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 377/765] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 15:43:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:28.583853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadaeaa-5f18-4c9e-af1c-c377df7af431 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.584141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadaeaa-5f18-4c9e-af1c-c377df7af431 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.584416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadaeaa-5f18-4c9e-af1c-c377df7af431 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.589978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 378/766] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:28.589978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadaeaa-5f18-4c9e-af1c-c377df7af431 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.614949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.651585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.676475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae7754-8b27-4fde-9422-aaf52baf1f26 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.710443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadaeaa-5f18-4c9e-af1c-c377df7af431 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.720892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.721554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.721836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.722114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.735384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.736897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.753568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-44f7c790-4870-4f0a-8192-d4307c424c2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:28.753747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:28.754647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:28.755727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:28.755727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:28.755727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:28.755899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 389/767] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:28.757020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.757286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.757406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.757569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.758031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.758491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.759464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:28.760007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.761367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:28.767806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 379/768] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 15:43:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:28.768520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.778403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:28.779635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:28.859635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 390/769] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:28.860861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:28.901923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355840d6-6623-42c7-9886-d4f06ce4b9fa None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:28.918704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.933750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:28.971475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:28.973320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:28.975830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:28.976343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:28.977207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:28.978282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:28.981388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.003776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 380/770] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:29.010137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.031439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbbe4c-f851-456a-8875-9367af1d5d0a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.084952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.086039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.086527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.087014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.088002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] RBAC: Adding query filter params (interface=public, service_id=b2a4215c900d440cbda95870a5a9cd18, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:29.088538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.090608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.100611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 391/771] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:43:28 2022] GET /identity/v3/endpoints?service_id=b2a4215c900d440cbda95870a5a9cd18&interface=public®ion_id=RegionOne => generated 194 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:29.107388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.197504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1480400-90c7-4dcb-9d67-abf08779a289 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.234495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.235324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.235632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.235905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.237563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:29.246198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.246504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828897-dcb5-4364-ab92-e20992e1b86c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.248056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:29.260682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:29.261935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:29.262834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:29.265223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 381/772] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:29.266205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.288565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.289356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.289747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.290096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.290883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.292894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.300586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 392/773] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/services => generated 772 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:29.304323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.330447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:29.481279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d76831-073a-469f-91c3-13b6e5eb1f36 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.493904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482d85a5-095f-4beb-b77b-cbc46bad93b2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.524253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.525311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.525311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.525481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.525963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.527358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.534945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.535722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.536004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.536275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.536972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:29.537623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8e529944-7201-4157-9d85-0bb78645d9c7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:29.538647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:29.539643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:29.541055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:29.546489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95580e5f-7d2c-4dde-9a9f-8f038be0ac14 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.547761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 382/774] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/services/network => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:29.547761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e984d-3505-41eb-807e-dc39d617775c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.547761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e984d-3505-41eb-807e-dc39d617775c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.547761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e984d-3505-41eb-807e-dc39d617775c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.547761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 383/775] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:29.550118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e984d-3505-41eb-807e-dc39d617775c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.558722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 393/776] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:29.559894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.560180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.560452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.580560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:29.581799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:29.598446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:29.662677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 394/777] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:29.668236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.827232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17b380-2152-4034-8f15-f258a5f5693e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.868217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.868968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.869246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.869520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.870246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=4411226c5c7c49d987880e64e363a3ff) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:29.870551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.872005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.878179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 395/778] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/role_assignments?user.id=4411226c5c7c49d987880e64e363a3ff&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:29.888315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:29.892981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e984d-3505-41eb-807e-dc39d617775c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:29.933582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:29.934194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:29.934470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:29.934742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:29.935378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] RBAC: Adding query filter params (name=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:29.935709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:29.937133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:29.942468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 384/779] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 392 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:29.949014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:29.971716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:29.996528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:30.083760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1174edb-0a37-4453-bdce-c1e69d587334 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.130499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.131123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.131403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.131748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.132507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] RBAC: Adding query filter params (type=network) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:30.132816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:30.134314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:30.145902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811af61-3f0e-42e3-a3d3-66c62430c528 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.166355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 396/780] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:30.176247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:30.188361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.188999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.189274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.189539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.203514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:30.209106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:30.210057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:30.211512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 385/781] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:30.223568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:30.267631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:30.326188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:30.352142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e824746-2656-4958-8f09-b378b26c3c53 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.354106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aef20d-6c8e-40d4-b721-28166424f7f1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.388897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86800880-e517-4959-9fae-ec748e20026a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.389602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86800880-e517-4959-9fae-ec748e20026a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.389877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86800880-e517-4959-9fae-ec748e20026a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.390149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86800880-e517-4959-9fae-ec748e20026a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.390959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86800880-e517-4959-9fae-ec748e20026a None admin] RBAC: Adding query filter params (interface=public, service_id=83ae4bee2acd4f85852f77759dd3c4c3, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:30.391065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.391272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86800880-e517-4959-9fae-ec748e20026a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:30.391648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.391930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.392202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.392689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86800880-e517-4959-9fae-ec748e20026a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:30.392869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:30.393172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:30.394638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:30.398536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 397/782] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:43:30 2022] GET /identity/v3/endpoints?service_id=83ae4bee2acd4f85852f77759dd3c4c3&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:30.403856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 386/783] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:30.408633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86800880-e517-4959-9fae-ec748e20026a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:30.409161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:30.524412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:30.546019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06af908-7e35-47f9-941b-d7433a1704cb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.606341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86800880-e517-4959-9fae-ec748e20026a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.614117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.614761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.615036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.615307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.640587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:30.642030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:30.644645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.645515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.645997 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.646469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.647429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:30.650351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:30.664470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 398/784] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:30 2022] GET /identity/v3/services => generated 772 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:30.681418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:30.682472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:30.683391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:30.683391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:30.683391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:30.688382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:30.692100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 387/785] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:30.697185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:30.835825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2e8cc9-257b-428d-899b-8a77b3b69c8c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:30.876739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:30.877715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:30.878215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:30.878702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:30.879590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:30.880108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:30.881744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:30.895672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 399/786] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:30.903682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.041106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb4789-f1ab-41c3-ad2e-5a75c79094f1 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:31.078374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:31.079002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.079280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.079570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.087370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:31.088763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:31.090603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f1a81700-cb0d-48a9-be3a-1c9453a68e94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:31.101265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:31.102253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:31.104118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:31.105481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 388/787] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:31.110481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.126363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440fd5-911e-4eec-a304-77e5c12ad1da None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.126649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440fd5-911e-4eec-a304-77e5c12ad1da None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.127032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b440fd5-911e-4eec-a304-77e5c12ad1da None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.128619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 389/788] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:31.133606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.134380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.134664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.153481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:31.154695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:31.246813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:31.272440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 390/789] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:31.301833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2cdfc9-edf8-44d4-98f8-4ed9f3314067 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:31.339287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:31.339934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.340215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.340488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.341226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:31.341534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:31.342976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:31.361617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 400/790] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 15:43:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:31.368688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.507637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219bc58-70a0-472c-be4c-2ed23974fd84 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:31.534984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:31.543483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:31.548273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.548567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.548848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.566413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:31.567964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:31.586538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:31.588139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:31.589400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:31.589400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:31.589400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:31.589400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 391/791] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:31.596744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.635100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.733024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158191cc-ae79-465c-8e46-2ba7cfc39844 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:31.757277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:31.775119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:31.775692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.775977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.776252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.776903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:31.777209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:31.778584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:31.799331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 401/792] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:31.803930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.833680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5294d-f68b-46fa-8a06-161f15204448 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:31.899410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b7871-2888-4a16-b454-d7a5d6f8855c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:31.900695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b7871-2888-4a16-b454-d7a5d6f8855c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.900973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b7871-2888-4a16-b454-d7a5d6f8855c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.901240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b7871-2888-4a16-b454-d7a5d6f8855c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.918194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8cf064-b870-4ec1-b5d0-b2ff73ed4723 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:31.928203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b7871-2888-4a16-b454-d7a5d6f8855c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a06b7871-2888-4a16-b454-d7a5d6f8855c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:31.933949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:31.934924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:31.935810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:31.940004 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 392/793] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:31.941343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea9838-ae07-40ea-9a50-d404f5ddcab0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.941624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea9838-ae07-40ea-9a50-d404f5ddcab0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.941903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea9838-ae07-40ea-9a50-d404f5ddcab0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.943391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 393/794] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:31.947991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea9838-ae07-40ea-9a50-d404f5ddcab0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:31.953919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:31.954566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.954851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.955130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.964395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:31.966510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-adaccd7c-bceb-4854-8ac2-519450558bfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:31.984636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:31.986350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 402/795] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:31.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:31.990212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:31.998232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.011428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:32.012967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:32.059443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.061207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea9838-ae07-40ea-9a50-d404f5ddcab0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.107576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 403/796] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:32.107576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.108245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.109262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.109776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.110252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.111349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:32.112100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.113912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.131353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 394/797] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:32.136888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.215294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26d110-c56e-4b55-8c37-07147defebe4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.255443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.256089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.256371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.256645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.257762 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:32.258070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.259822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.273819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 404/798] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:32.282354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.334733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9df260-05b8-435a-96d2-5a1ce29a7d0b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.336699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.345977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.375967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.377141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.377656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.378032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.389758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.391745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:32.408152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:32.409280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:32.410383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:32.410383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:32.410383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:32.411881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 395/799] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:32.417222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.512721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affeb29-24ac-4ea2-805b-cca02613e8ac None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.528722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c845fe-ae8e-4cee-bc13-6f9a63ad9e2b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.560457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.561884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.562429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.562954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/f92a1a83dc5a4ababdebb85429de82b0/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.565434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=f92a1a83dc5a4ababdebb85429de82b0, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.567115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.567939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.568552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.568919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.569273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.569950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:32.570359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.572159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec4f37-a09a-4ed2-a08c-4d13ce2914d3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.582600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 405/800] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:32 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/f92a1a83dc5a4ababdebb85429de82b0/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:43:32.583603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.599844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 396/801] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:32.603192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580903f0-d235-44c0-86b2-0a67dd0106fc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.603506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580903f0-d235-44c0-86b2-0a67dd0106fc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.603748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580903f0-d235-44c0-86b2-0a67dd0106fc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.605606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 397/802] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:32.606103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580903f0-d235-44c0-86b2-0a67dd0106fc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.623595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.705965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.719315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb309d0-1a17-4978-abe1-53a201c1776b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.721361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580903f0-d235-44c0-86b2-0a67dd0106fc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.760200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.760583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.760819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.761090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.763795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.765128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.765628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.766103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.766893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.768531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.770432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.771916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-147ab06c-978d-4e84-84ed-9921b034a70c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:32.778520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:32.779769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 07 15:43:32.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:32.782737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 406/803] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:32.784034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.785162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:32.785278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:32.786374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:43:32.787630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:32.788312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.788398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 398/804] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:32.791569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.811040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:32.812564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:32.888489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:32.895743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 407/805] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:32.897002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:32.920107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42f689-c883-4486-a298-34b9a17a2868 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:32.960502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:32.961857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:32.962381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:32.962799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:32.963698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:32.964142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:32.965800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:32.986207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 399/806] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:32.991936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.008753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.038729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016aa8f-475b-4675-9395-e96ddd886963 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.079979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.080878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.081263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.081743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.082486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:33.082894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.084431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.089817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 408/807] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:33.100667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.132583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b5517-3ca7-4952-8593-ba2ced835a26 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.163460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.168739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.172137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.172137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.172137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.266479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ecb414-67ad-4d01-bf7e-c7b25282cd1e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.274423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.276436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.304813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.305427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.306343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.306695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.307431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:33.307898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.309274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.322544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 409/808] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:33.324415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.346252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.390890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 400/809] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 15:43:33.393019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.438998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.456550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40d3111-8b8e-468a-b6c0-186114f9ff13 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.497613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.498504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.498935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.499356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.510885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:33.517102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:33.518227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:33.519170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:33.521452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6137-7cc1-438d-a5d7-a87d0ad7d361 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.521609 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 410/810] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:33.522495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.566476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.567202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.567578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.567927 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.568662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.570095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.596111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 401/811] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:43:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:33.597224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.621086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.652816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449b4cdb-9cbe-449b-a7f9-0fe7d7a15f7b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.701149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.701813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.702101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.702433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.719170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.720720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.742080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f1e7f9b5-4eaa-4177-baed-40ea9dced65b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:33.742383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:33.743423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:33.744394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:33.744394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:33.744394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:33.747713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 411/812] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:33.749304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f7bfe-4275-4de2-930d-3b443f64edb3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.750035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f7bfe-4275-4de2-930d-3b443f64edb3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.752674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f7bfe-4275-4de2-930d-3b443f64edb3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.756310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 412/813] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:33.758222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f7bfe-4275-4de2-930d-3b443f64edb3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.770882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2830f1c-7428-4fd7-93ec-1fe019c08d83 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.814633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.815247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.815529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.815847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.816512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:33.816820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.818183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7db11f-44ef-4e64-908e-b774dff69d16 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.832647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 402/814] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:33.833954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.834262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.834520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.854235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:33.856510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:33.881007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f7bfe-4275-4de2-930d-3b443f64edb3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:33.907915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:33.921932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:33.922638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:33.922921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:33.923191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:33.923865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:33.924175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:33.925542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:33.937236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 403/815] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:33.938553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 413/816] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:33.940405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:33.944640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.075862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:34.102071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9238a6b-e29b-4070-84cb-e72fc41fd30f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.124915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b2ba0-c699-4d6a-a3eb-6682f187e0d5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.160237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.160972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.161228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.161415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.172133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.173606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:34.192929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:34.193878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:34.194810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 414/817] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:34.196982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.197896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.198239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.198576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.206164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.211672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.216579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:34.244181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:34.245204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:34.246130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:34.246130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:34.246130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:34.254542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 404/818] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:34.261204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.287460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:34.351335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc9329-3021-4963-b12c-556c32dc6593 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.384045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b9de2-13af-4694-831b-81899d325b70 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.396242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.396860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.397399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.397679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.398376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:34.398681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.400929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.421585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 415/819] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:34.421585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.423567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:34.429314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.431526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.432110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.432759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.433692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:34.434202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.435818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.453712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 405/820] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:34.454921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.521220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362d478b-db91-4f85-ae47-c6101a792aee None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.522735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:34.561245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.562574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.563058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.563354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] PATH_INFO: `/v3/services/network` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.563972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.565471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.569182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a28844-ea34-4cac-a5e4-f014b1b3c4ba None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:34.575693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:34.576801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 07 15:43:34.577681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:34.581799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 416/821] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:34.582643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.611524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.612707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.613573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.614367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.615652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=8a9090ab0ffa4b7eafcc742880237229) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:34.616245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.619307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.635180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 406/822] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/role_assignments?user.id=8a9090ab0ffa4b7eafcc742880237229&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:34.635180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.749456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a366c-9823-43a8-941f-e38245dfd352 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.760613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:34.832014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb378be7-98db-4ef5-be48-f80dbde83f83 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:34.834308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.834955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.835231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.835500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.851512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.853600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:34.866713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:34.867725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:34.868785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 417/823] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:34.874298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.892801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:34.893864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:34.894466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:34.894946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:34.895868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] RBAC: Adding query filter params (name=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:34.896388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:34.898128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:34.903219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:34.905567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 407/824] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:34.910931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:34.994194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69b2d68-260a-4f3f-9524-45a488ab2dc8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.033733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.034345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.034626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.034893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.035654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:35.035959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.037388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.052330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac6626-5097-48e2-b239-0c21f744bf3c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.061954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 418/825] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:35.069498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.092633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:35.100691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.101296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.101570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.101837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.102974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] RBAC: Adding query filter params (type=network) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:35.103286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.104916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193aac13-0f11-466c-ad2e-80f8185addbf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.111481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 408/826] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:35.116891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931842b3-a324-48cc-89d1-acb3f217fba2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.117117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931842b3-a324-48cc-89d1-acb3f217fba2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.117386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931842b3-a324-48cc-89d1-acb3f217fba2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.119412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 409/827] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:35.124468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931842b3-a324-48cc-89d1-acb3f217fba2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.170857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:35.192533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c9f39d-9ed7-4b3c-9b12-b7767d0b6983 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.230031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.230994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.231536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.232014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.244137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.246275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.247151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931842b3-a324-48cc-89d1-acb3f217fba2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-491f3a15-c067-4154-b5e7-fe22d106295c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:35.263525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:35.264542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:35.265460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:35.265460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:35.265460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:35.271679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 419/828] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:35.273282 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.273757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.274216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.284697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.285384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.285738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.286087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.286831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.288319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.296387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:35.297901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:35.311763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 410/829] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:43:35 2022] POST /identity/v3/endpoints => generated 337 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:35.313009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.363113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:35.394797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 420/830] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:35.487723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82aa069-0d37-43b9-a3ba-5b97f42976ba None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.526896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.527509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.527816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.528088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.528729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:35.529030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.530383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.543464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 411/831] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:35.549690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.627603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:35.678261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e35484e-310a-432d-b2f5-23313c1d3ebb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.700828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.716704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.717454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.717736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.718035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.725533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.726991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:35.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:35.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:35.742492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 421/832] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:35.747956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.809789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:35.812938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f135-c9fd-47aa-a9e5-b41695d9112c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.851583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.852254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.853102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.853381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.863322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3390026-4c8c-4aff-839e-1397537c15da None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:35.866031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:35.871210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:35.873443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:35.875518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 412/833] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:35.883642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.890698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:35.905323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:35.906199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:35.906479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:35.906754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:35.907449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:35.907793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:35.909470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:35.922859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 422/834] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:35.933122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:35.987532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9739c08f-efca-4b9c-b192-372d7409fc3c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.025339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.025950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.026351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.026627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.027293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:36.027722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.029099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7856e23b-a9b1-49ac-a921-fb61b92ba05d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.037894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 413/835] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:36.043783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9a3af-14ed-4a93-a05a-e1b22fe61b91 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.044073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9a3af-14ed-4a93-a05a-e1b22fe61b91 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.044351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a9a3af-14ed-4a93-a05a-e1b22fe61b91 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.045899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 414/836] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:36.046606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9a3af-14ed-4a93-a05a-e1b22fe61b91 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.051696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44886ebb-4648-4e07-b79e-948d22309cdc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.074663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:36.087506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38663e18-10e6-457e-8c86-cd25ca941f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.088202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38663e18-10e6-457e-8c86-cd25ca941f80 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.088485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38663e18-10e6-457e-8c86-cd25ca941f80 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.088760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38663e18-10e6-457e-8c86-cd25ca941f80 None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/4411226c5c7c49d987880e64e363a3ff/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.103754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38663e18-10e6-457e-8c86-cd25ca941f80 None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=4411226c5c7c49d987880e64e363a3ff, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.105466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38663e18-10e6-457e-8c86-cd25ca941f80 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.119596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 423/837] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:35 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/4411226c5c7c49d987880e64e363a3ff/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:43:36.120253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.120540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.120816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.151537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:36.153254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:36.185538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a9a3af-14ed-4a93-a05a-e1b22fe61b91 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.192666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:36.244047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.245256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.245543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.245822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.257159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 424/838] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:36.263689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.263689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:36.281129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:36.282372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:36.283367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:36.283367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:36.283367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:36.285391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 415/839] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:36.291927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.401518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e5636-2e75-427d-a593-eed2b91b452f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.436357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.436889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.437167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.437437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.438075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:36.438376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.439826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.451972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 425/840] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:36.457285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.525000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.538357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:36.553891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:36.569467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9adfa7-cdff-4462-961b-0fe4d8084261 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.607322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.608124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.608725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.609100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.617410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.619526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:36.632882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:36.634095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 07 15:43:36.635282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:36.641552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 416/841] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:36.642640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.653928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec71fc-454e-460b-be9b-380fe2f87599 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.700686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.701647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.702333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.702757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.704301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.707820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:36.719851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:36.721116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:36.722172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:36.727594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 426/842] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:36.730246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.765045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd48a3-b91b-4386-82aa-ba0ebe87a8fd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.801577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.802420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.802847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.803257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.804260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:36.804731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.806884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.808786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:36.822692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:36.828918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 417/843] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:36.834174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.846912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a010b6-2b9b-4ab9-a826-b0bef1619bf3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:36.886458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18991fef-6823-4569-a117-069e675328d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:36.887360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18991fef-6823-4569-a117-069e675328d2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:36.887788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18991fef-6823-4569-a117-069e675328d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:36.888163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18991fef-6823-4569-a117-069e675328d2 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:36.889014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18991fef-6823-4569-a117-069e675328d2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:36.889421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18991fef-6823-4569-a117-069e675328d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:36.890891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18991fef-6823-4569-a117-069e675328d2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:36.901366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 427/844] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:36.902752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18991fef-6823-4569-a117-069e675328d2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:36.959714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de0614-1309-4e7d-a74a-fcc1fb780e4f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.004012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430aba85-9fef-4ad5-bce9-6d903ac097e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.004837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aba85-9fef-4ad5-bce9-6d903ac097e9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.005214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aba85-9fef-4ad5-bce9-6d903ac097e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.005621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430aba85-9fef-4ad5-bce9-6d903ac097e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.038565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18991fef-6823-4569-a117-069e675328d2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.077528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.078331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.078719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.079096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.079896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:37.080310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.081830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.082116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:37.093687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 428/845] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:37.100887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430aba85-9fef-4ad5-bce9-6d903ac097e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.103200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430aba85-9fef-4ad5-bce9-6d903ac097e9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.235780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 418/846] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:43:37.241130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.268425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:37.277100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a6280-6d46-4ec6-a846-85dd84f5c66e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.277636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a6280-6d46-4ec6-a846-85dd84f5c66e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.278054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3a6280-6d46-4ec6-a846-85dd84f5c66e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.280515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 419/847] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:37.286508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.287042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.287456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.308153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:37.309351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:37.361258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5088c-0f18-4b62-821b-9026089885b0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.384329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 420/848] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:37.404067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.404797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.405147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.405493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.417984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.419623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:37.433839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:37.435022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:37.436167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:37.436167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:37.436167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:37.437660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 429/849] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:37.444558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.470586 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:37.554994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17f071-fde9-4cd4-82a5-f30cfeac3f00 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.590002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.590613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.590946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.591222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.591888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:37.592187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.593544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.609758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 421/850] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:37.610672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.675685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.682468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:37.723936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6c7031-af84-4482-9e42-9c9b28cda4a5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.760892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.761662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:37.762670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.763093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.763497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.771756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.773551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.781941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec665825-ff72-492e-83ea-2e660c87e55b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:37.785911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:37.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:37.788003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:37.790727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 430/851] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:37.795931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.823129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.823821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.824168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.824681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.825236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.826660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:37.835744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:37.836699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:37.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:37.841869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 422/852] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:37.842740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.918432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f970c8-c927-4034-9112-1a2a340909ab None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.947349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:37.949783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63b1ff-432d-4fb1-b567-b126f602a930 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:37.958375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.959449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.959953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.960439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.961456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:37.961988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.963842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:37.978277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 431/853] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:37.985122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:37.995293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:37.995964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:37.996240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:37.996510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:37.997146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:37.997445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:37.999378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:38.010032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 423/854] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:38.011342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:38.028971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:38.119503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fb42dc-69c1-4bf6-b7ac-9a65f8f8369e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:38.130592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328e168b-8720-4df5-9193-ba1ea883cddc None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:38.162610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:38.163462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:38.163904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:38.164418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:38.165632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=f92a1a83dc5a4ababdebb85429de82b0) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:38.166085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:38.167792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69aacb8-5b78-4a7a-9f5e-7c04df62476b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:38.174689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 432/855] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:37 2022] GET /identity/v3/role_assignments?user.id=f92a1a83dc5a4ababdebb85429de82b0&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:38.185237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:38.186278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:38.186831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:38.187304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:38.188213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:38.188743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:38.190480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b5b9b5-a91f-4646-bd61-9add4a4cb51a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:38.197287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 424/856] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:38.398095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b642ce2-8a97-4440-9848-4d76cfd9aa14 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:38.398289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b642ce2-8a97-4440-9848-4d76cfd9aa14 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:38.398568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b642ce2-8a97-4440-9848-4d76cfd9aa14 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:38.400755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 433/857] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:38.406410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:38.406872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:38.406950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:38.427729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:38.428785 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:38.507660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 425/858] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:38.770797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b642ce2-8a97-4440-9848-4d76cfd9aa14 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:38.824133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:38.878325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b642ce2-8a97-4440-9848-4d76cfd9aa14 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:38.913970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:38.915568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:38.916049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:38.916515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:38.927246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:38.932373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:38.933309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:43:38.934172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:38.936788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 434/859] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:43:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:38.943576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:38.945810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:39.051468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21df12a0-ebc5-4cf4-9889-d88b5c1317ca None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.086617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.087779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.088052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.088314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.088969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:39.089264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:39.090619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:39.099219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 426/860] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:43:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:39.105300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:39.217584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195deea6-d7db-49aa-bce4-ffd951a489d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.231346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:39.256062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.256731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.257083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.257364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.269299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:39.270865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-8e65f084-0086-45f6-8c63-c06ada5bce6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:39.287826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:39.288859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:39.289770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:39.289770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:39.289770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:39.294604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 435/861] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:39.296334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:39.380275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d1e21-32f7-466f-a859-03b7a20687f8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.380567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d1e21-32f7-466f-a859-03b7a20687f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.380762 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d1e21-32f7-466f-a859-03b7a20687f8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.381236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:39.383392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 436/862] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:39.387905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.388248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.388530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.409246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:39.410774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:39.419747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b808ca83-e014-4445-9e36-6c3897e9ef8c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.457045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.457943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.458432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.458890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.459736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:39.460222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:39.461811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:39.475327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 427/863] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:39.481348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:39.497623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 437/864] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:39.584384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6d1a83-d2cd-4069-b080-382ac6e3c636 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.612125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:39.621367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.622003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.622281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.622557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.629924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:39.631422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:39.643200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:39.644179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:39.645728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 428/865] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:43:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:39.650509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:39.743797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:39.753109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1da7b-eeee-42ba-b324-029944334cb5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.787025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.787696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.787989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.788269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.789009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:39.789320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:39.790853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:39.812544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 438/866] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 15:43:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:39.817727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:39.819210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:39.929054 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe018b8-9fd2-4bcd-bc33-c84fbfb9c40c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.939822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caeaafc-31c1-416b-947e-7a0b03517a82 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:39.966311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.967264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.967886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.968381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.976827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:39.977508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:39.977790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:39.978065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:39.978726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:39.980508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:39.993545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:40.007574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 429/867] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:43:39 2022] POST /identity/v3/services => generated 251 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:40.060008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:40.061562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:40.181169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 439/868] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 15:43:40.188737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:40.241037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:40.303976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab97269a-0bb6-4820-be1e-f6a481bd93db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:40.338711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:40.339355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.339663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.339938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.351263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:40.353279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-9873e159-5ac3-4fd7-a054-6cc15499e4aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:40.370928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:40.372002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:40.372952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:40.372952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:40.372952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:40.375678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 430/869] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:40.378079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:40.436524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:40.498957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2478063-8f04-42c0-b0fd-67793b95dbef None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:40.532347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612367b0-616f-4773-a35f-d710cfd3385e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.532862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612367b0-616f-4773-a35f-d710cfd3385e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.532862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612367b0-616f-4773-a35f-d710cfd3385e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.534612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 431/870] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:40.536259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:40.537079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.537551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.538012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.538973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:40.539463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:40.539781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8b45b-a258-4623-b57e-bff2b8a2bc3b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.540108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8b45b-a258-4623-b57e-bff2b8a2bc3b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.540395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8b45b-a258-4623-b57e-bff2b8a2bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.541922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:40.561321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 440/871] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:40.562292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a2a8b45b-a258-4623-b57e-bff2b8a2bc3b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:40.563219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:40.563507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a2a8b45b-a258-4623-b57e-bff2b8a2bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:40.634236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 432/872] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:40.637257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290363e-1b52-46ba-bec6-73db6e210352 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.637548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290363e-1b52-46ba-bec6-73db6e210352 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.637825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c290363e-1b52-46ba-bec6-73db6e210352 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.644524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 433/873] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:40.646282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.646576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.646857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.671414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:40.672644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:40.681745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7730031f-5182-46b4-b3b4-257bb184ea4c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:40.691782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:40.698804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:40.733989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:40.734685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.735009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.735292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.744507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:40.746238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:40.756211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 434/874] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:40.761927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:40.762810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:40.763787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:40.767850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 441/875] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:40.772502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:40.874224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a2fe6-8830-467d-8c96-b52c9699e80b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:40.903296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:40.909888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:40.910501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:40.910779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:40.911052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:40.911777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:40.912085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:40.913513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:40.931512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 435/876] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:40.932859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:40.950778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:41.025310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb196e9-33a0-4b20-bbd1-afb32ba5a83e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:41.042413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e3e7b-916a-4648-870c-23911954315d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:41.063794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:41.064695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:41.065081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:41.065364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:41.066029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:41.067405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:41.077500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:41.078075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:41.078354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:41.078623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:41.079353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=4411226c5c7c49d987880e64e363a3ff) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:41.079692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:41.081110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:41.084794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 442/877] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:43:40 2022] POST /identity/v3/services => generated 269 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:43:41.085754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:41.090377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 436/878] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:40 2022] GET /identity/v3/role_assignments?user.id=4411226c5c7c49d987880e64e363a3ff&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:41.134080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:41.230721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30a483-c418-47b6-bcaf-1e92d3cc4330 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:41.275822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:41.276483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:41.276765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:41.277042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:41.277673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:41.279087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:41.290162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:41.291217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:41.292123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 443/879] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:41.297867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:41.401315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:41.424518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc5088-309d-4be0-8345-df11237ce2a3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:41.463639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:41.464244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:41.464519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:41.464786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:41.465419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:41.465717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:41.467067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef75c71-a80a-4146-a159-fb7fa51410ce None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:41.472201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 437/880] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:41.477959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:41.571599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:41.581916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3e95b-31bc-47b6-8be9-8accd8b8fe1d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:41.620890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:41.621514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:41.621795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:41.622067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:41.622707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:41.623012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:41.624412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619c4ba-838e-46d9-86c2-3e570ba87808 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:41.629583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 444/881] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:42.317087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5a76-36fb-4dd4-b9df-377aada08f86 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:42.318105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5a76-36fb-4dd4-b9df-377aada08f86 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:42.318601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c5a76-36fb-4dd4-b9df-377aada08f86 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:42.320468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 438/882] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:42.326583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:42.327108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:42.327636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:42.346449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:42.347993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:42.423725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 445/883] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:42.689562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c5a76-36fb-4dd4-b9df-377aada08f86 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:42.800337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c5a76-36fb-4dd4-b9df-377aada08f86 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:42.835963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488971e-013c-4924-8093-46358b21eeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:42.837129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488971e-013c-4924-8093-46358b21eeba None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:42.837414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488971e-013c-4924-8093-46358b21eeba None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:42.837691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7488971e-013c-4924-8093-46358b21eeba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:42.838280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488971e-013c-4924-8093-46358b21eeba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:42.839675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7488971e-013c-4924-8093-46358b21eeba None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7488971e-013c-4924-8093-46358b21eeba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:42.848888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:42.849916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:42.850976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 439/884] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:42.858398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:42.948083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:42.963418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fa457-59b8-41f7-ae37-280927c5fed0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:42.999860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:43.000888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.001441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.001926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.002793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:43.003310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:43.004920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:43.014710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 446/885] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:43.016307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488971e-013c-4924-8093-46358b21eeba None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:43.026930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:43.123649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7488971e-013c-4924-8093-46358b21eeba None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:43.162012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:43.162640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.162922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.163198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.163872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:43.164182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:43.166155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:43.171860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 440/886] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:43.177867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:43.288403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78994d48-3c79-4c80-8669-4d0ce967430d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:43.324011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:43.324989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.325499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.325994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.326904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] RBAC: Adding query filter params (interface=public, service_id=adfca9cc80b14195adf829ebc2c25773, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:43.327416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:43.329058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cba194-2329-40b0-ab87-f0402ddac634 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:43.339730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 447/887] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:43:43 2022] GET /identity/v3/endpoints?service_id=adfca9cc80b14195adf829ebc2c25773&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:43.343031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:43.368901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99640fea-68ff-42e4-8100-b855cf8d3342 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.369221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99640fea-68ff-42e4-8100-b855cf8d3342 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.369459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99640fea-68ff-42e4-8100-b855cf8d3342 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.371284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 448/888] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:43.377375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3385b02-5845-4976-90ae-4e27e5aaa7b2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.377683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3385b02-5845-4976-90ae-4e27e5aaa7b2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.378027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3385b02-5845-4976-90ae-4e27e5aaa7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.397425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f3385b02-5845-4976-90ae-4e27e5aaa7b2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:43.398713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f3385b02-5845-4976-90ae-4e27e5aaa7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:43.408011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:43.427205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:43.461486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517ae01-08dc-46b7-b27f-cedbc238c432 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:43.476677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 449/889] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:43.485790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d14e9-da69-4a68-b5a7-d2ea384560fc None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.486686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d14e9-da69-4a68-b5a7-d2ea384560fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.487611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d14e9-da69-4a68-b5a7-d2ea384560fc None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.489947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 450/890] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:43 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:43.495581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df9bb9-1ea6-4640-a025-d2f618326ae3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.495867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df9bb9-1ea6-4640-a025-d2f618326ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.496142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9df9bb9-1ea6-4640-a025-d2f618326ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.501961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:43.503083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.504359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.504819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.505611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:43.507274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:43.514641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 441/891] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:43 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:43:43.514792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a9df9bb9-1ea6-4640-a025-d2f618326ae3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:43.519799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a9df9bb9-1ea6-4640-a025-d2f618326ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:43.599495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 451/892] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:43.753172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:43.861634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c57f49d-2fcb-42e1-92ac-802d8e8b071f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:43.869031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997971f-ab20-4f73-8984-ce63f102578c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.869517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997971f-ab20-4f73-8984-ce63f102578c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.869604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1997971f-ab20-4f73-8984-ce63f102578c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.871164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 452/893] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:43.876373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.876660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.876938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.895510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:43.896745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:43.901559 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:43.902460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:43.902944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:43.903416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:43.904232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:43.906211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:43.909141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:43.919111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:43.920206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:43.926291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 442/894] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Sep 7 15:43:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:43.927183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:43.966911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 453/895] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:43.972624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:43.976255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.043299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dab489-d55a-482c-a3fc-514b6146d98e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.080717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.080938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e0851a-9fce-4dd8-be14-fb2bf016fa65 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.081827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.082415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.082979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.094409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:44.101250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:44.102765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:43:44.103668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:44.106369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 443/896] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 15:43:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:44.112398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.120486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.121100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.121380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.121652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.122288 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:44.122593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.124324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.133251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 454/897] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:44.134115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.181028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.204518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.232419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adea446-2b8b-4c41-9e64-ffb9a61f4714 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.241084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6a7b3-e7fb-47da-8e98-29cea2526574 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.270836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.272006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.272551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.272913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.273624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:44.273934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.275638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.275866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.276264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.276549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.276827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.277588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:44.278114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.279293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.290104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 455/898] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:44.290299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 444/899] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:44.290404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.290500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.395560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f4806-26a2-4a29-9db9-a8b3ce2faf5b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.401469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c79ef4-f3cb-400e-92d6-caaa7997598b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.423763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.429644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.436624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.437437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.437718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.437994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.438649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.439724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.440123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.440355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.440638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.440911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.452019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.453481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.454334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 456/900] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:43:44 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:44.454983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:44.482833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:44.484084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:44.485216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:44.485216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:44.485216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:44.491886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 445/901] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:44.493347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.583916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc45ac6-7737-4a40-adf9-ebc95756e332 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.620863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a13464-5274-47e2-acfc-4431fd8c05ee None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.622458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.623085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.623365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.623683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.624388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] RBAC: Adding query filter params (interface=public, service_id=609e9b45b2694807bd1eb475c9dc5e97, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:44.624700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.626110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.632651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 457/902] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/endpoints?service_id=609e9b45b2694807bd1eb475c9dc5e97&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:44.638305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.665199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.665821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.666094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.666361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.667023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:44.667328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.669518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.681663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 446/903] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:44.687031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.709525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.723879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:44.781006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb677e0-cd80-47d4-a867-146bb373cda0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.802123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21782ab4-9b77-4575-a8a8-ccec1562cbcb None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:44.828297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.829366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.829907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.830187 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.830805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.833120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.840556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:44.840748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 458/904] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/services => generated 1497 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:43:44.841192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:44.841474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:44.841747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:44.850287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:44.851784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:44.866085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:44.867134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:44.868230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 447/905] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:44.879931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:44.989353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f60d5-fffc-4ef0-9025-cca62e600db2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:45.005582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:45.033037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:45.033676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.033938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.034211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.034949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:45.035256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:45.036730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:45.055024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 459/906] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 15:43:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:45.061065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:45.157487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:45.166290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43f7e3b-cd51-43b4-873c-485844a1a864 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:45.202622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:45.203234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.203512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.203810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.275324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:45.276717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:45.372372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:45.382833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 448/907] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:43:45.389586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:45.492622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff8ea9-2d16-404a-8852-923db280e690 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:45.532312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:45.533296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.533798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.534296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.545640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:45.547380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:45.563423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:45.564448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:45.565376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:45.565376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:45.565376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:45.567182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 460/908] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:45.573218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:45.691325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe03c256-ca55-431f-9eb4-eefbf2d573f4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:45.732078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:45.732686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.732963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.733229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.733870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:45.734171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:45.735620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dda59a-6b69-4f25-a5f4-1d2fa69c9bd0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:45.747726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 449/909] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:45.753242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:45.779625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:45.836408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2187-6768-4fd2-bbb0-59b84c2f39a9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.837559 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2187-6768-4fd2-bbb0-59b84c2f39a9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.838036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d2187-6768-4fd2-bbb0-59b84c2f39a9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.839959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 450/910] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:45.845878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.846514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.847011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.859908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd8ec21-8726-4165-87fe-ea9e91b2d523 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:45.869115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:45.871230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:45.903451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:45.904162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:45.904461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:45.905042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:45.915213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:45.916700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:45.928039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:45.931266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:45.932185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:45.933171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 461/911] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:45.940559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:45.963562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 451/912] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:46.042102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:46.059784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19febc4-8f46-4cfb-b437-3dd15dd0da38 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:46.096128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:46.096662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.096938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.097207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.097952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:46.098255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:46.099706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:46.112919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 462/913] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:46.118932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:46.227778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4df40-73d4-40f1-a143-6976ea602f3f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:46.263865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:46.264476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.264751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.265018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.265741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] RBAC: Adding query filter params (role.id=8f5ed297a5184b2fa9424b2f5429027f, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=4411226c5c7c49d987880e64e363a3ff) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:46.266048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:46.267473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:46.273605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 452/914] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:46 2022] GET /identity/v3/role_assignments?user.id=4411226c5c7c49d987880e64e363a3ff&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=8f5ed297a5184b2fa9424b2f5429027f => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:46.278102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:46.390873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0881fd73-2d00-430a-a81f-5547e6e3538d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:46.429688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:46.430768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.431257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.431777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.432585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:46.434175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:46.444022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:46.445102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 07 15:43:46.445992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:46.454111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 463/915] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:46.455281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:46.515030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:46.571417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fbbe37-54db-48e9-8c7b-84471532f8f0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:46.610946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:46.611621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.611858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.612128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.612817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:46.613117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:46.614525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e634162-899e-4f28-b671-7852e0cec655 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:46.620210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 453/916] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:46.625655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:46.642884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:46.687878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd0af1-5d41-4c55-88b6-ccce56f8f7ef None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.688231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd0af1-5d41-4c55-88b6-ccce56f8f7ef None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.688431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd0af1-5d41-4c55-88b6-ccce56f8f7ef None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.691040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 454/917] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:46.695052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.695339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.695657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.714870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:46.716190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:46.749712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ff4b5b-c46c-4463-87b9-d6b646e17413 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:46.787095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:46.787838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.788190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.788746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.789231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:46.790127 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:46.791652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:46.801642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 464/918] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Sep 7 15:43:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:46.802209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 455/919] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:46.804528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:46.875903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:46.907998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb23428-7f4c-4d7a-afbd-bc0cba0a528a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:46.943345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d51f9-615a-426d-adf3-c8f6a4ea9c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:46.943985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d51f9-615a-426d-adf3-c8f6a4ea9c2f None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:46.944263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d51f9-615a-426d-adf3-c8f6a4ea9c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:46.944532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d51f9-615a-426d-adf3-c8f6a4ea9c2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:46.945193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d51f9-615a-426d-adf3-c8f6a4ea9c2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:46.947163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d51f9-615a-426d-adf3-c8f6a4ea9c2f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:46.970654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 465/920] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:43:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:47.071617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.116921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:47.148146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb571-6ab3-4f5f-a9e3-c774f2caab05 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.148492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb571-6ab3-4f5f-a9e3-c774f2caab05 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.148712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb571-6ab3-4f5f-a9e3-c774f2caab05 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.150875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 466/921] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:47.156363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.156648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.158498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.177549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:47.178760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:47.205038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece81361-421d-48b3-a1df-2c63b56d2dc1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.256651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.257331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.257614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.257891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.258555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.260047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.266595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 467/922] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:47.269994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:47.271049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:47.271916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:47.275317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 456/923] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:47.281131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.343455 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:47.389661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f08a86-c919-4a98-8405-cf0635ac274c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.426014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.426557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.426839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.427113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.427792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:47.428097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.429538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.438892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 468/924] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:47.440980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.543763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:47.546983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12330271-0cbc-425e-8d76-380371f54300 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.564358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.587196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.587842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.588123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.588396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.589035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:47.589340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.591462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.599620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 457/925] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:47.604263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.702078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9046c0d8-821a-4946-9e22-497703baed39 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.731395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:47.732847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b116fe-26b0-48a8-92ac-797a07e461d9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.747319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.748068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.748479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.748761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.749350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.751426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:47.763105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:47.764237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:47.765158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 469/926] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:47.772467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.781043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.781653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.781928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.782197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.782899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] RBAC: Adding query filter params (interface=public, service_id=4cc8af61c9e749c28bfecf0501224268, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:47.783204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.784594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.793774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 458/927] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/endpoints?service_id=4cc8af61c9e749c28bfecf0501224268&interface=public®ion_id=RegionOne => generated 194 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:47.799108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:47.829335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:47.905527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c004888-beba-4cb0-b74f-39682163944c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.911642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8380b-0d1b-49a0-98e1-29ebd7661323 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:47.940961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.941587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.941844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.942116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.942725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.944123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.950790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 459/928] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:43:47.966467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:47.967029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:47.967313 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:47.967628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:47.968318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:47.968629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:47.970115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:47.976457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 470/929] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:47.982699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:48.089596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:48.100066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ec2d8-5ec4-4d0a-a58d-aa90141b53f7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:48.138224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:48.138834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:48.139111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:48.139379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:48.140045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:48.140346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:48.141726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee5a91-e426-47ed-b680-a459c2d88b58 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:48.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 460/930] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:48.152821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:48.213795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:48.265057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90246a53-1b8b-43a7-8815-6dee6c01ca5a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:48.301911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c53fe-084c-418d-b803-6616656498bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:48.302526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c53fe-084c-418d-b803-6616656498bc None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:48.302806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c53fe-084c-418d-b803-6616656498bc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:48.303078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c53fe-084c-418d-b803-6616656498bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:48.303781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c53fe-084c-418d-b803-6616656498bc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:48.305201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c53fe-084c-418d-b803-6616656498bc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:48.329476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 471/931] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:43:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:48.621545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c158d-1c85-4999-8c1d-02db2c0ef7a9 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:48.621917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c158d-1c85-4999-8c1d-02db2c0ef7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:48.622294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c158d-1c85-4999-8c1d-02db2c0ef7a9 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:48.624058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 461/932] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:48.629406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:48.629808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:48.630188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:48.650322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:48.651528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:48.732251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 472/933] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:49.002290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c158d-1c85-4999-8c1d-02db2c0ef7a9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.113971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c158d-1c85-4999-8c1d-02db2c0ef7a9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:49.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9116f84-b425-4670-9614-3051c28b7370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:49.155212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116f84-b425-4670-9614-3051c28b7370 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.155665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116f84-b425-4670-9614-3051c28b7370 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.156084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9116f84-b425-4670-9614-3051c28b7370 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.168787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9116f84-b425-4670-9614-3051c28b7370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e9116f84-b425-4670-9614-3051c28b7370 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:49.174130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:49.175278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:49.180980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 462/934] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:49.183187 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.215844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:49.247985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c449-dfec-4b46-b95e-9805f602a7a3 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.248319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c449-dfec-4b46-b95e-9805f602a7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.248527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08c449-dfec-4b46-b95e-9805f602a7a3 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.250174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 463/935] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:49.254988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.255207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.255479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.278371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:49.279832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:49.299476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753f9a4-8d4d-403b-a906-4941261112ce None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:49.305710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:49.339125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:49.339780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.340051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.340311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.340976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:49.341281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:49.342651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:49.351172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 473/936] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:49.359598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.365645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 464/937] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:49.457730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:49.469452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839bc22-8340-445e-9d31-2ce9576965a2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:49.504654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:49.505636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.506185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.506670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.517649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:49.519364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:49.534390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:49.535382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:49.536356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:49.536356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:49.536356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.538382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 474/938] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:49.544474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.638661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.651807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb79c95-f1e2-4170-ad25-e4c7c26a90a5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:49.687665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:49.688276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.688552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.688821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.689466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:49.689766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:49.691115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:49.703580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 465/939] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:49.708868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.721253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:49.764805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d33d37-8a06-40b9-8736-3d9904b55f79 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:49.801061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:49.801698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.801974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.802245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.802901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:49.804501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:49.807959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING keystone.server.flask.common [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 07 15:43:49.826615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 15:43:49.826615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 15:43:49.826615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [parameters: {'id': '408eb9b138b848189e44999ee4de32e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 15:43:49.826615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 15:43:49.831806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f4908b-739b-49fa-8db9-1a1039312d4b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 408eb9b138b848189e44999ee4de32e5.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 408eb9b138b848189e44999ee4de32e5. Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 15:43:49.843687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 07 15:43:49.844575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Sep 07 15:43:49.850573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 15:43:49.851495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 15:43:49.853984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 15:43:49.856348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'id': '408eb9b138b848189e44999ee4de32e5', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.858641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:49.861025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 408eb9b138b848189e44999ee4de32e5. Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 475/940] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Sep 7 15:43:49 2022] POST /identity/v3/projects => generated 262 bytes in 207 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.864458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:49.892537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:49.893514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:49.894007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:49.894579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:49.903501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:49.905343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:49.918891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:49.919928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:49.920879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:49.928349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 466/941] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:49.929745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:49.968351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb909e-aa50-42ce-9910-06da35114dc7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.003170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:50.006564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.007276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.007689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.007957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.016633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.018342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:50.030484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:50.031405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:50.032393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:50.035886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 476/942] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:50.037582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.049476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa25278-ae30-4b17-a248-b25b1d82d3dd None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.090420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.091508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.092076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.092568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.093527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:50.094120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.095899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.104479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:50.123601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 467/943] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 15:43:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:50.125448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.146333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc7670a-928a-49ce-8f59-1d3d09afc41d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.186604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.187405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.187754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.187858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.188569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:50.188865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.190359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0df6e04-0147-4635-a646-947e0e7b571e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.219897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 477/944] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:50.246084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94527c7c-3a0f-40c0-931f-8e639afe8b23 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.274473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:50.286761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.287414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.287727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.288000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.288600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233735a-5f56-466d-b768-c2d68f533bc1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.288889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233735a-5f56-466d-b768-c2d68f533bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.289168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5233735a-5f56-466d-b768-c2d68f533bc1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.290761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 478/945] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:50.296444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.297061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.297663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.299862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.301496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:50.319162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:50.320407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:50.321407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:50.321407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:50.321407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:50.323490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:50.325349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 468/946] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:50.325650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:50.326672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.423587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 479/947] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:50.449007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538312b8-d9c9-4a9d-9990-44d72ef8e725 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.486709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.487349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.487684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.487960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.488636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:50.488943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.490438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a24a23-c9e2-4f1c-8006-7ef244caed98 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.508342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 469/948] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:50.511141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.568775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:50.600561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd47f3-b971-4f50-a6e3-a87deb88c906 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.600857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd47f3-b971-4f50-a6e3-a87deb88c906 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.601115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd47f3-b971-4f50-a6e3-a87deb88c906 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.602655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 470/949] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:50.607650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.607935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.608208 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.623867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1ccffc-b170-4dfb-891b-6da8b207f044 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.626978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:50.628183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:50.660070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:50.668130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.669048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.669523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.669978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.679425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.681632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:50.697104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:50.698059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:50.698972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:50.699602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 471/950] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:50.699823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.705732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 480/951] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:50.708617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.836024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb559ae-d443-4548-999e-4d3bd358244c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.841112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e195491d-7984-405d-9c20-223bd3fb25aa None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:50.854247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:50.880100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.880990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.881366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.881734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.882416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.883970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.884240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:50.884864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:50.885152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:50.885429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:50.886427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:50.886744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:50.889055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:50.894294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:50.895450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:50.896515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:50.897374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 472/952] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:50.902880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.905386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 481/953] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:50.909945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:50.940146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:51.018761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843359b-d0c3-4b3e-b505-b8fcac66b49d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.029930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6abab3-1802-43ff-945a-bf89c597661f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.055117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.055759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.056040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.056306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.056940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:51.057241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.058783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.063945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 473/954] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:51.064541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:51.071452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.072156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.072441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.072713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/4411226c5c7c49d987880e64e363a3ff/roles/8f5ed297a5184b2fa9424b2f5429027f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.074874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=4411226c5c7c49d987880e64e363a3ff, role_id=8f5ed297a5184b2fa9424b2f5429027f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.076337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.103683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 482/955] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:43:50 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/4411226c5c7c49d987880e64e363a3ff/roles/8f5ed297a5184b2fa9424b2f5429027f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:43:51.103683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:51.139202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:51.204086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:51.214905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee7607-6e28-4060-aadc-e134742fd9db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.244051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379e376-784d-4c4e-b902-d3f555b57b2c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.263158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.264426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.265031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.265632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.266584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.268361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:51.279198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:51.280589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:43:51.281511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:51.283776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 474/956] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:51.291460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:51.294506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.295435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.295746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.296024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.297651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:51.297960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.299418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.306183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 483/957] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:51.312128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:51.426045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18874403-1a72-4fda-aa25-7c35e16a1463 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.430721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:51.438868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58997f22-a4dc-492c-98fa-9e3b7d4f87c8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.463572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.464329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.464715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.465138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.465799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:51.466374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.467721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.473173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 475/958] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:51.477622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ec523c-383b-421c-984d-9d9b1bf88bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.478416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec523c-383b-421c-984d-9d9b1bf88bdb None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.478872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec523c-383b-421c-984d-9d9b1bf88bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.478990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:51.479393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ec523c-383b-421c-984d-9d9b1bf88bdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.480602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec523c-383b-421c-984d-9d9b1bf88bdb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.482129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ec523c-383b-421c-984d-9d9b1bf88bdb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.506425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:51.507362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 484/959] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:43:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:51.603791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ba089-635b-419e-81eb-9f458ea4c850 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:51.655509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:51.656157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:51.656432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:51.656770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:51.657414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:51.657713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:51.659078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e0470-7601-4718-b0f0-89160fe638a2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:51.664829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 476/960] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:52.575812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59609df-b61e-4cb6-9f4b-89d4e9c17a84 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:52.576790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59609df-b61e-4cb6-9f4b-89d4e9c17a84 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:52.577284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59609df-b61e-4cb6-9f4b-89d4e9c17a84 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:52.579092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 485/961] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:52.584749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:52.585529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:52.586012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:52.605278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:52.606741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:52.681894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 477/962] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:52.949023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59609df-b61e-4cb6-9f4b-89d4e9c17a84 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:53.056442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59609df-b61e-4cb6-9f4b-89d4e9c17a84 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:53.091360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:53.092391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.092881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.093362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.094235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:53.095987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:53.112228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.sql.core [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 15:43:53.112228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 15:43:53.112228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [parameters: {'user_id': '0558d42bed474ba7a172f950a3cc79b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 15:43:53.112228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81459) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Sep 07 15:43:53.119852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 15:43:53.120777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application yield resource Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application next(self.gen) Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.rollback() Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.force_reraise() Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise self.value Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application session.commit() Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 07 15:43:53.121792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._prepare_impl() Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.session.flush() Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._flush(objects) Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application compat.raise_( Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application flush_context.execute() Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rec.execute(self) Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 07 15:43:53.122709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 07 15:43:53.125828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result = self._query(query) Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application conn.query(q) Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application result.read() Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 07 15:43:53.133027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [parameters: {'user_id': '0558d42bed474ba7a172f950a3cc79b1', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:53.135128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 07 15:43:53.137112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:53.139332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 486/963] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Sep 7 15:43:52 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:53.144713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:53.159625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:53.257516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00aa3b-c06b-4e9e-8b43-db5cf1238f30 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:53.298037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:53.299049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.299529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.300026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.308978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:53.310748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7ed8454f-85bc-4cd4-85b9-ef8bbe0f4d78 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:53.322925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:53.323956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 07 15:43:53.324928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:53.331671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 478/964] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:43:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:53.333957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:53.352115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:53.392245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875adab-523c-4ac8-8d0a-bb3d26cc14c0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.393031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875adab-523c-4ac8-8d0a-bb3d26cc14c0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.393530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3875adab-523c-4ac8-8d0a-bb3d26cc14c0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.395688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 479/965] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:53.402629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4511fa5-3667-4c19-9c4b-93e26b101d72 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.402913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4511fa5-3667-4c19-9c4b-93e26b101d72 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.403185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4511fa5-3667-4c19-9c4b-93e26b101d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.421877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d4511fa5-3667-4c19-9c4b-93e26b101d72 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:53.423086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d4511fa5-3667-4c19-9c4b-93e26b101d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:53.464560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1d561-3e03-43c9-a885-86059d48e265 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:53.502041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:53.503018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.503484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.503926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.504865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:53.505302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:53.507066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34e30f4-287e-41e9-824c-41db603fc845 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:53.509245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 480/966] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:53.551648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 487/967] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Sep 7 15:43:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:53.725127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805234-fa26-4a2e-be90-7feb9ca49499 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.725673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805234-fa26-4a2e-be90-7feb9ca49499 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.725673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94805234-fa26-4a2e-be90-7feb9ca49499 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.727991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 481/968] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:53.734191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6628d-2cf6-42b0-9a47-8d8941a9d9f8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.734725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6628d-2cf6-42b0-9a47-8d8941a9d9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.734725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6628d-2cf6-42b0-9a47-8d8941a9d9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.754839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0cc6628d-2cf6-42b0-9a47-8d8941a9d9f8 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:53.756433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0cc6628d-2cf6-42b0-9a47-8d8941a9d9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:53.779959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805234-fa26-4a2e-be90-7feb9ca49499 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:53.783806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:53.844525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 488/969] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:53.858629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158a13-326d-4d7f-aaf5-b9ae2c618b14 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.858927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158a13-326d-4d7f-aaf5-b9ae2c618b14 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.859177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0158a13-326d-4d7f-aaf5-b9ae2c618b14 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.860589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 489/970] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:53.865976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.866237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.866511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.873799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:53.887200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:53.888431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:53.915382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94805234-fa26-4a2e-be90-7feb9ca49499 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:53.953922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:53.954544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:53.954824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:53.955090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:53.963584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 490/971] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:53.965562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:43:53.970372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:53.971301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:53.972207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 482/972] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 15:43:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:53.979830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.076249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1be3f7-05ba-446e-8ab2-d0d417887d6e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.105733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.111436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.112191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.112572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.112945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.113708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:54.114150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.115296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.115754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.129232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 491/973] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:43:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:54.130356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.194663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.238406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c40ace-9073-4c2a-a2ef-6b2c270a205c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.240887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5f4f4-adee-4274-9a26-8763fde5d7fb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.272301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.272932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.273242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.273542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.274187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.274402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.275130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.275509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.275658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.276003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:54.284313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:54.285463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:54.286455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 483/974] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:54.287447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.291522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.293013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:54.307681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:54.308851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:54.309980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:54.309980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:54.309980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:54.309980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 492/975] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:54.309980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.357017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.389324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc7c0ee-eb7e-4f82-9a8f-428fbdbc76b9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.409700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91554-8fbb-4537-a08c-9a7088762901 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.414217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.423661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.424256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.424531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.424798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.425436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.426788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.440285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 484/976] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:43:54 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:54.441059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.454290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.454993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.455275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.455567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.456235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:54.456542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.457977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.464466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 493/977] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:54.470078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.569061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59266441-428d-4e0e-89d4-1977d78def85 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.588994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd27c9-aab0-4b47-ad2a-948f38eb62e8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.596868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.611694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.613065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.613065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.613065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.614129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:54.614129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.615959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.627437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.628221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.629199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.629367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 485/978] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:54.629473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.629746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:54.630150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.631663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.634308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.637328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 494/979] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:54.643072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.686824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.746248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7077c99-a0b1-4b3c-afa1-3759af05e5e9 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.747644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f207d4f-eb72-4e35-8cbc-ccdb648ca6d7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.782747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.783124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.783357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.783667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.783771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.783931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.784027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.784295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.784643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] RBAC: Adding query filter params (interface=admin, service_id=4cc8af61c9e749c28bfecf0501224268, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:54.784951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.786350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.796419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 495/980] 10.222.0.17 () {56 vars in 1249 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/endpoints?service_id=4cc8af61c9e749c28bfecf0501224268&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:54.796602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.796602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.797469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:43:54.808009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:54.809209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 07 15:43:54.810393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:54.814590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 486/981] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:54.815420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:54.837045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.908917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47176fec-d3fc-4629-b3bd-f431ba859b4e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.924086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76501f-20d6-4a3c-9cbd-7eec3f5d3f92 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:54.932183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:54.948492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.949136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.949412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.949690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.950292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.951692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.967575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 496/982] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:43:54.970036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:54.970672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:54.970948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:54.971219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:54.972013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:54.972319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:54.973802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:54.994990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 487/983] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:55.000595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:55.103931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:55.108101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3abe5c9-e423-461f-a118-d1918abb82df None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:55.144165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:55.144781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:55.145061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:55.145335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:55.223008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:55.224750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:55.319770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:55.326194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 497/984] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:43:54 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:43:55.331387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:55.438758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f0dba-246c-4af7-bb6c-0bc8d333699b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:55.476298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:55.477417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:55.477700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:55.477970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:55.488543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:55.489981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:43:55.504238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:55.505189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:43:55.506236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:43:55.506236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:43:55.506236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:55.506236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 488/985] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:55.512706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:55.613607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a573d58-5406-4020-9cbe-17abc842950d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:55.647151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:55.647785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:55.648064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:55.648333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:55.649483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:55.649784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:55.651148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:55.662744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 498/986] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:43:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:55.668122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:55.689050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:55.776167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968d71-88f6-4071-9272-96152e4c6a11 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:55.809921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:55.810549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:55.810827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:55.811099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:55.817683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:55.819112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:43:55.830709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:55.831633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:55.832637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 489/987] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:55.839389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:55.900184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:55.942666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246a87-430e-470f-b2b4-02baa16991c2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:55.975292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863e224-7796-485e-944b-09aed729041a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:55.975931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863e224-7796-485e-944b-09aed729041a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:55.976215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863e224-7796-485e-944b-09aed729041a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:55.976490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863e224-7796-485e-944b-09aed729041a None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:55.977697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863e224-7796-485e-944b-09aed729041a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:55.978005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863e224-7796-485e-944b-09aed729041a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:55.979433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863e224-7796-485e-944b-09aed729041a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:55.996618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 499/988] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:43:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:55.997841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:56.074158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:56.097780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ad5b0-50d0-40df-a14c-030d90671471 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:56.131898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:56.132499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:56.132778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:56.133078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:56.133801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] RBAC: Adding query filter params (role.id=8f5ed297a5184b2fa9424b2f5429027f, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=4411226c5c7c49d987880e64e363a3ff) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:56.134107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:56.135525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa32d19-0240-4351-b9f5-a2989ebcbead None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:56.147575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 490/989] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:43:55 2022] GET /identity/v3/role_assignments?user.id=4411226c5c7c49d987880e64e363a3ff&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=8f5ed297a5184b2fa9424b2f5429027f => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:56.626163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68075-1f42-40dd-875b-0bf9d0248ec1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:56.626558 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68075-1f42-40dd-875b-0bf9d0248ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:56.626740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68075-1f42-40dd-875b-0bf9d0248ec1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:56.628140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 500/990] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:56.633696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:56.633987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:56.634261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:56.652505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:56.653697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:56.721092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 491/991] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:56.989010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68075-1f42-40dd-875b-0bf9d0248ec1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.098584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68075-1f42-40dd-875b-0bf9d0248ec1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.115862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:57.133929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.134568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.134850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.135120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.135794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.137173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-39039fc6-d35f-4052-bd3a-c02decc22bb0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:57.145847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:57.146832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:43:57.148071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:57.155315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 501/992] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:43:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:57.155457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.161411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54886978-8385-4a2b-baaa-44655d3a5371 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.161703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54886978-8385-4a2b-baaa-44655d3a5371 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.161983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54886978-8385-4a2b-baaa-44655d3a5371 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.163456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 502/993] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:43:57.168360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.168645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.168964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.187688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:57.188953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:57.233178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:57.275617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 503/994] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:43:57.275798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac811f6-eaa9-4f97-bf34-7dc708431c5f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.304082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.304683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.304997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.305269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.305900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] RBAC: Adding query filter params (name=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:57.306198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.307562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.316980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 492/995] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:57.318201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.423794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c0073-2e7e-4840-8468-aae7c92ed6d6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.438368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:57.460690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466db325-f806-4344-9481-c7e24f773eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.461337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466db325-f806-4344-9481-c7e24f773eda None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.461617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466db325-f806-4344-9481-c7e24f773eda None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.461889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466db325-f806-4344-9481-c7e24f773eda None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.462511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466db325-f806-4344-9481-c7e24f773eda None admin] RBAC: Adding query filter params (type=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:57.462816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466db325-f806-4344-9481-c7e24f773eda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.464200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466db325-f806-4344-9481-c7e24f773eda None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.471911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 504/996] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:57.474896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.535479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466db325-f806-4344-9481-c7e24f773eda None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.587196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9ded7-9011-48b6-838a-0d9dffbb4cec None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.621530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.622141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.622420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.622809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.623505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] RBAC: Adding query filter params (interface=public, service_id=7616571c5652424486b99247885ac2fb, region_id=RegionOne) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:57.623843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.625191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.626629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:57.635848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 493/997] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/endpoints?service_id=7616571c5652424486b99247885ac2fb&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:57.637014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.654335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466db325-f806-4344-9481-c7e24f773eda None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.689661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.690297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.690580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.690859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.691892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.693292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:57.702798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:57.707840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:43:57.708727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 505/998] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:43:57.710447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.746018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb017f71-992f-44b2-98c7-5f825137cb17 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.755318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:57.788389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.789970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.790244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.790510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.791524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.793520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.800803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 494/999] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/services => generated 1727 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:43:57.815722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5bad7-da94-4a9c-a5ed-b8af430988d8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:57.857720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:57.858812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:57.859312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:57.859895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:57.860804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:57.861324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:57.863058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:57.869748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 506/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:57.875497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:57.943150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:57.985796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2412c79-5314-4ed8-a566-7dcca23e5689 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:58.023383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48594115-0945-477e-9858-b544204b28d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:58.024123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48594115-0945-477e-9858-b544204b28d5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:58.024408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48594115-0945-477e-9858-b544204b28d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:58.024684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48594115-0945-477e-9858-b544204b28d5 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:58.025319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48594115-0945-477e-9858-b544204b28d5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:58.025624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48594115-0945-477e-9858-b544204b28d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:58.026979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48594115-0945-477e-9858-b544204b28d5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:58.032106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 495/1001] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Sep 7 15:43:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:58.037917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:58.141308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bc768-ce83-4f89-92cf-aba4235df85b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:58.143601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:58.187283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff27ab2-19c8-4846-adf2-9fa33ed630cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:58.187933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff27ab2-19c8-4846-adf2-9fa33ed630cb None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:58.188217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff27ab2-19c8-4846-adf2-9fa33ed630cb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:58.188493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff27ab2-19c8-4846-adf2-9fa33ed630cb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:58.189179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff27ab2-19c8-4846-adf2-9fa33ed630cb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:58.190552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff27ab2-19c8-4846-adf2-9fa33ed630cb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:58.206635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 507/1002] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:43:58 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:43:58.348571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1c6a-d1bd-4689-8db8-7dc0dcc03d54 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:58.349130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1c6a-d1bd-4689-8db8-7dc0dcc03d54 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:58.349130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff1c6a-d1bd-4689-8db8-7dc0dcc03d54 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:58.350640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 496/1003] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:58.357010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:58.357293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:58.357575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:58.377126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:58.378326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:43:58.455943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 508/1004] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:58 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:43:58.720818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff1c6a-d1bd-4689-8db8-7dc0dcc03d54 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:58.823880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff1c6a-d1bd-4689-8db8-7dc0dcc03d54 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:58.848945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:58.859887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:58.860558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:58.860837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:58.861104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:58.861679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:58.863194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:43:58.872046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:43:58.873077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:43:58.873937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 497/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Sep 7 15:43:58 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:43:58.880612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:58.970979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:43:58.982329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0213d6e-7bb0-4896-937c-2fca8c777829 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:59.017492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:59.017976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:59.018261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:59.018536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:59.019164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] RBAC: Adding query filter params (name=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:59.019472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:59.020963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400530fd-7406-4c7e-83fe-ca6097673bbf None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:59.026536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 509/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:58 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:43:59.033396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:43:59.140027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4df0c76-0ff4-4bfc-839c-1d0ac0a577a7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:43:59.184051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:43:59.184660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:59.184943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:59.185216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:59.185852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] RBAC: Adding query filter params (type=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:43:59.186160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:43:59.187524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad6616c-e2cd-40c9-91f9-6e349bf84f0a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:43:59.192146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 498/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:43:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:43:59.972561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32866bb-ee09-4744-a99e-938fb46b55a9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:59.973082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32866bb-ee09-4744-a99e-938fb46b55a9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:59.973082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32866bb-ee09-4744-a99e-938fb46b55a9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:59.974571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 510/1008] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:43:59.979819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f39ff8-7a2e-4be3-a8be-b37727f977ed None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:43:59.980220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f39ff8-7a2e-4be3-a8be-b37727f977ed None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:43:59.980400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f39ff8-7a2e-4be3-a8be-b37727f977ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:43:59.998711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-35f39ff8-7a2e-4be3-a8be-b37727f977ed None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:43:59.999923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-35f39ff8-7a2e-4be3-a8be-b37727f977ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:00.049110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:00.069680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 499/1009] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:43:59 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:00.335100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32866bb-ee09-4744-a99e-938fb46b55a9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:00.371710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739b614-5366-4ec2-9803-c1eeb60c417f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.371996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739b614-5366-4ec2-9803-c1eeb60c417f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.372461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739b614-5366-4ec2-9803-c1eeb60c417f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.373792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 500/1010] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:44:00.378971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.379248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.379518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.398450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:00.400442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:00.442440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32866bb-ee09-4744-a99e-938fb46b55a9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:00.475969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 501/1011] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:00 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:00.482789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:00.483500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.483892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.484235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] PATH_INFO: `/v3/services/compute` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.484895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:00.486326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:00.495910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:00.496899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:00.497784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 511/1012] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:44:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:00.502709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:00.603187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1d9e6-6fe5-45ab-8b0f-68ad8fde1f99 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:00.618277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:00.639961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:00.640564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.640842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.641110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.641737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:00.642040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:00.643396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:00.653342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 502/1013] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:44:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:00.654268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:00.674679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:00.742798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:00.758435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb622cc8-2a9e-421a-83ac-7168a1b3c3ca None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:00.796190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:00.797094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.797374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.797642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.798276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:00.798578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:00.799987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:00.804932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 512/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Sep 7 15:44:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:00.810704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:00.847018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e050d-0296-491a-983f-438d3b9694d8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:00.881653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:00.882206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.882484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.883040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.887760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:00.890019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:00.892251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 503/1015] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:00.900183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:00.909559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:00.932697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ac6c1c-f0aa-457a-992f-aa72d7a27ad8 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:00.949316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:00.968986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4538a5-6fb8-49fc-aa24-c020fe33a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:00.969605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4538a5-6fb8-49fc-aa24-c020fe33a174 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:00.969972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4538a5-6fb8-49fc-aa24-c020fe33a174 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:00.970164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4538a5-6fb8-49fc-aa24-c020fe33a174 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:00.970832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4538a5-6fb8-49fc-aa24-c020fe33a174 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:00.972240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4538a5-6fb8-49fc-aa24-c020fe33a174 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:00.995898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 513/1016] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:44:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:44:01.012423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeecdb5e-3a8c-4add-9b2d-f935f9eaf420 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:01.059216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2f9443-8aa2-4c5f-84bc-ec19f4af4ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:01.059880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f9443-8aa2-4c5f-84bc-ec19f4af4ca8 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:01.060162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f9443-8aa2-4c5f-84bc-ec19f4af4ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:01.060436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2f9443-8aa2-4c5f-84bc-ec19f4af4ca8 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:01.061113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f9443-8aa2-4c5f-84bc-ec19f4af4ca8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:01.062597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2f9443-8aa2-4c5f-84bc-ec19f4af4ca8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:01.094059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 504/1017] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:44:00 2022] POST /identity/v3/projects => generated 311 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:44:01.373591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3887-0997-4bc7-9bed-cf3fa3b93e75 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:01.373878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3887-0997-4bc7-9bed-cf3fa3b93e75 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:01.374157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce3887-0997-4bc7-9bed-cf3fa3b93e75 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:01.375556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 514/1018] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:01.380862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c07e0b-aa2a-42b2-9ab8-41c24afd6c71 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:01.381153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c07e0b-aa2a-42b2-9ab8-41c24afd6c71 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:01.381426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c07e0b-aa2a-42b2-9ab8-41c24afd6c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:01.399994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-18c07e0b-aa2a-42b2-9ab8-41c24afd6c71 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:01.401178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-18c07e0b-aa2a-42b2-9ab8-41c24afd6c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:01.474492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 505/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:01 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:01.749734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce3887-0997-4bc7-9bed-cf3fa3b93e75 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:01.852498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce3887-0997-4bc7-9bed-cf3fa3b93e75 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:01.886727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d189f-a8d2-447e-aeaa-8898344d522c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:01.887348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d189f-a8d2-447e-aeaa-8898344d522c None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:01.887664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d189f-a8d2-447e-aeaa-8898344d522c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:01.887940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d189f-a8d2-447e-aeaa-8898344d522c None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:01.888601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d189f-a8d2-447e-aeaa-8898344d522c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:01.890010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d189f-a8d2-447e-aeaa-8898344d522c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:01.902285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:01.915718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 515/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:44:01 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:03.262682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afc8d-d030-44b4-a711-80990396b286 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.263212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afc8d-d030-44b4-a711-80990396b286 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.263212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9afc8d-d030-44b4-a711-80990396b286 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.264583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 506/1021] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:03.270028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.270419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.270585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.289439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:03.290642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:03.317564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5538bd-c573-4aab-8bdc-0d74c8546fd5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.317850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5538bd-c573-4aab-8bdc-0d74c8546fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.318108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5538bd-c573-4aab-8bdc-0d74c8546fd5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.319434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 507/1022] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:44:03.324798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.325159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.325524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.343432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:03.344844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:03.363328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:03.369875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 516/1023] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:03.418049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 508/1024] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:03.636591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:03.694961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:03.740610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4367b-3b36-4968-9583-36cb548fdfd2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:03.767173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:03.779328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:03.779996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.780277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.780549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.790876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:03.795834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:03.796798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:03.797683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 517/1025] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 15:44:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:03.803062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:03.811275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9429da-1875-42c0-a0b0-49507f36e27b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:03.854931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:03.855610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.855982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.856262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.857385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:03.858829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:03.864987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 509/1026] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:03.866313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:03.892836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:03.913058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3acd6c6-4c45-4fcd-965f-5e17f81e0a63 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:03.948131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:03.948671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:03.948952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:03.949225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:03.949900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:03.950210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:03.951607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:03.960140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 518/1027] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:44:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:03.966621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:03.996271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0026ae74-2a3c-470b-a279-c462d3a7be52 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.017332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.037828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0c8a5-60f6-46c4-8aba-4fdc557f7b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.038444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0c8a5-60f6-46c4-8aba-4fdc557f7b12 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.038714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0c8a5-60f6-46c4-8aba-4fdc557f7b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.039582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0c8a5-60f6-46c4-8aba-4fdc557f7b12 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.040456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0c8a5-60f6-46c4-8aba-4fdc557f7b12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.041702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0c8a5-60f6-46c4-8aba-4fdc557f7b12 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.075574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 510/1028] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:44:03 2022] POST /identity/v3/users => generated 285 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:44:04.079341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c3519-b181-4fc9-a7fc-6d8ded1c28b1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.090231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f1b5e-6e4d-417a-ae05-e1b29b4a3a1d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.091380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f1b5e-6e4d-417a-ae05-e1b29b4a3a1d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.091952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3f1b5e-6e4d-417a-ae05-e1b29b4a3a1d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.094709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 511/1029] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:44:04.101119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.102339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.102811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.120249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.120914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.121196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.121475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.127455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:04.128736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:04.135426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.136938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:04.154300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:04.155512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:04.156597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:04.156597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:04.156597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:04.158348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 519/1030] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:04.164600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.210388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 512/1031] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:04.283210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.288118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1becd2e4-e500-4f55-a658-e29835f67e45 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.330799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.331411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.331729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.332003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.332648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:04.332952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.334319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.347040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 520/1032] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:04.352669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.364094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.456850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c3f03-e56c-49a7-b746-293ac5385b72 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.486063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.493939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6c285-a504-4343-8220-128107296d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.494578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6c285-a504-4343-8220-128107296d3f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.494857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6c285-a504-4343-8220-128107296d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.495129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6c285-a504-4343-8220-128107296d3f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.504242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6c285-a504-4343-8220-128107296d3f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.505682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6c285-a504-4343-8220-128107296d3f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-03a6c285-a504-4343-8220-128107296d3f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:44:04.518184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:04.519164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:44:04.520187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:04.523708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 513/1033] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:04.524955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6c285-a504-4343-8220-128107296d3f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.594040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f411ae-49a0-41a5-a579-6fc47508f25b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.597521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.634848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6c285-a504-4343-8220-128107296d3f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.642933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.643587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.643872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.644150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] PATH_INFO: `/v3/services/image` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.644731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.646095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:04.655622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:04.656671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:04.657544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 521/1034] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:04.662426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.674664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.675283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.675591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.675876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.676609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:04.676919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.678353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.689037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.699588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 514/1035] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:04.702956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.773350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f092c3-f0bd-4bdd-8455-d6446f8331af None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.806591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7494fb-709e-46c1-8631-9774957fd2ea None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.812999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.813626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.813911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.814615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.815267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] RBAC: Adding query filter params (name=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:04.815606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.817368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.829003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 522/1036] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:04.830224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:04.833207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.842080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.842691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.842969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.843241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.919778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.921221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.926196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:04.944820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333815f4-e02e-4aec-b66e-abf8c668a2d5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:04.985371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:04.985983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:04.986249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:04.986532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:04.987140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] RBAC: Adding query filter params (type=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:04.987440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:04.988843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:04.998976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 523/1037] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:04.999849 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.033603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 515/1038] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 15:44:05.040138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.068079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:05.118769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93396c88-9a8c-45d6-a206-ffdf21bca1ed None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.152785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.153629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.154059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.154472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.155315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] RBAC: Adding query filter params (interface=public, service_id=5db395efb1a34a8aa5554c35a5dc0710, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:05.155806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.157339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.160240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd1fe-146f-4501-89b7-d67b7737adc3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.163138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 524/1039] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:44:04 2022] GET /identity/v3/endpoints?service_id=5db395efb1a34a8aa5554c35a5dc0710&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:05.168824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.185144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:05.205049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.205878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.206309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.206753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.218447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.220233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:05.235989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:05.237235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:05.238366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:05.238366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:05.238366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:05.245474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 516/1040] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:05.246597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.268961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2d340-7f90-407c-af33-6109043dbfa7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.309733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.310348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.310545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.310821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.311431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.312827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.319294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 525/1041] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:44:05 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:44:05.344584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:05.371806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ae6ab5-6de7-413e-b140-9419acec944f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.409175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.409804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.410086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.410360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.411020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:05.411327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.412757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.430965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 517/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:05.432314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.519152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:05.543640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54da503-cbcd-4c57-8a67-4763fb4a537a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.580604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.581164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.581451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.581730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.588413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.589858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:44:05.601184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:05.602647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:05.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 526/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:44:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:05.608939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.717091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073fda2f-376f-4e85-92d9-4e6d293374b5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.750938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.751605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.751857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.752129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.752816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:05.753122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.754541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87cab5f-8659-42db-9cf7-c146e59e9065 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.761766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:05.768247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 518/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:44:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:05.774628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:05.876692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3b44e-483d-4d7d-aa7a-ecfc54e9c4c3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:05.910270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:05.910880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:05.911162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:05.911434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:05.912202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] RBAC: Adding query filter params (role.id=8f5ed297a5184b2fa9424b2f5429027f, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=8a9090ab0ffa4b7eafcc742880237229) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:05.912514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:05.913975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aba386-a95e-4c1f-bce6-1d22fd3ae671 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:05.927570 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 527/1045] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:05 2022] GET /identity/v3/role_assignments?user.id=8a9090ab0ffa4b7eafcc742880237229&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=8f5ed297a5184b2fa9424b2f5429027f => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:06.324967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f4099-fade-4689-a8ad-909203328fcc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:06.325292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f4099-fade-4689-a8ad-909203328fcc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:06.325520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f4099-fade-4689-a8ad-909203328fcc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:06.326890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 519/1046] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:06.332124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a194e-0124-4f99-a618-cdb306819539 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:06.332582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a194e-0124-4f99-a618-cdb306819539 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:06.332663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a194e-0124-4f99-a618-cdb306819539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:06.338853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:06.352738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-945a194e-0124-4f99-a618-cdb306819539 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:06.353919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-945a194e-0124-4f99-a618-cdb306819539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:06.421509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 528/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:06.684714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f4099-fade-4689-a8ad-909203328fcc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:06.789871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f4099-fade-4689-a8ad-909203328fcc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:06.824488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:06.825102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:06.825374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:06.825639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:06.835651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:06.841210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:06.842172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:06.843036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:06.847594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 520/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:06.848561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a194e-0124-4f99-a618-cdb306819539 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:06.946751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a194e-0124-4f99-a618-cdb306819539 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:06.975726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:06.982724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:06.983337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:06.983642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:06.983918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:06.984578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:06.984883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:06.986307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:06.994833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 529/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:07.000773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:07.059853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:07.111900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d724c743-b209-4843-93bc-c671b6e6af2c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:07.146500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:07.147136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:07.147413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:07.147715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] PATH_INFO: `/v3/users/dbf5f475aa7346c0a42856e48c0c32ce` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:07.162487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] RBAC: Authorizing `identity:get_user(user_id=dbf5f475aa7346c0a42856e48c0c32ce)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:07.164488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:07.166316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 521/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:06 2022] GET /identity/v3/users/dbf5f475aa7346c0a42856e48c0c32ce => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:07.174755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:07.273911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe486687-b5d9-4fbc-8d0c-6fdb66aef391 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:07.307281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8189b8fa-fcd0-48bb-b4f8-96d7bc49df3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:07.307916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b8fa-fcd0-48bb-b4f8-96d7bc49df3c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:07.308209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b8fa-fcd0-48bb-b4f8-96d7bc49df3c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:07.308514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8189b8fa-fcd0-48bb-b4f8-96d7bc49df3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:07.366850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:07.375516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189b8fa-fcd0-48bb-b4f8-96d7bc49df3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:07.377004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8189b8fa-fcd0-48bb-b4f8-96d7bc49df3c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:07.484414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 530/1051] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:07 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 15:44:07.485577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:07.514539 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dd408-5467-4633-857c-8c86dfa2715b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:07.514823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dd408-5467-4633-857c-8c86dfa2715b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:07.515128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964dd408-5467-4633-857c-8c86dfa2715b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:07.516533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 531/1052] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:44:07.522262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:07.522552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:07.522831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:07.540878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:07.541996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:07.594272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9366d20e-53bb-4a18-9aab-e3985e5d5087 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:07.611829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:07.619597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 532/1053] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:07 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:07.630646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:07.631336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:07.631652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:07.631932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:07.633649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] RBAC: Authorizing `identity:get_project(project_id=83b5d6a06ffb49eab4c856a84207e111)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:07.635251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:07.642034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 522/1054] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:07 2022] GET /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:07.642963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:07.671045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:07.755369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453640a-78e5-4ccc-ba46-9e6d0c689c50 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:07.788663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:07.789834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:07.790977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:07.791206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:07.791934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=83b5d6a06ffb49eab4c856a84207e111, user.id=dbf5f475aa7346c0a42856e48c0c32ce) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:07.792240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:07.793654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda862e0-8156-453e-88d3-4cc2473b0c83 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:07.807808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 533/1055] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:07 2022] GET /identity/v3/role_assignments?user.id=dbf5f475aa7346c0a42856e48c0c32ce&scope.project.id=83b5d6a06ffb49eab4c856a84207e111&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:07.889878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.012870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28fe98c-64cd-4ca3-8aa4-e67694260b66 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:08.050226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:08.050863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.051179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.051402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] PATH_INFO: `/v3/services/image` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.052114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:08.053390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13250b-ff56-46e0-8b75-15fbcf9fe140 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.053525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:08.053717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13250b-ff56-46e0-8b75-15fbcf9fe140 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.053970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13250b-ff56-46e0-8b75-15fbcf9fe140 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.055454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 534/1056] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:44:08.060506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.060796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.061073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:08.064093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:08.065337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:08.066236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 523/1057] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Sep 7 15:44:07 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:08.072440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.085826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:08.087535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:08.127763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:08.177456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 535/1058] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:08 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:08.187071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb37453-d2b9-4f01-82de-1145d6fdc5f2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:08.222097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:08.222705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.222985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.223257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.223918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] RBAC: Adding query filter params (name=image) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:08.224221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:08.225579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:08.235381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 524/1059] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:08.236572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.256890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:08.357139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2d33eb-920e-4c76-b617-93cb2c4360d9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:08.393535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:08.394290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.394487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.394763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.395497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:08.395831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:08.397865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:08.403634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 536/1060] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:08.413547 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.448725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.541717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958de4c1-abf6-4e47-804b-1ed647961b5a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:08.551935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:08.559724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:08.575125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5394a9-3354-41c6-8742-b6d7a1efb0e2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:08.583905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:08.584521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.584799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.585073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.586318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:08.587729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:08.613557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 525/1061] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:44:08 2022] POST /identity/v3/endpoints => generated 327 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:08.627335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:08.628410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.628974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.629478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.643371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:08.649519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:08.650752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:44:08.651690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:08.659077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 537/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 15:44:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:08.660917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.808025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1e466-a5d2-4ec7-8f83-3c2907a4d12f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:08.843398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:08.844049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:08.844335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:08.844609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:08.845275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:08.845581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:08.846945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:08.856757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 526/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:44:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:08.862716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:08.966734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107d0de-0c9b-4d92-a3c4-980d2f1e77d5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.004403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:09.005045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:09.005324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:09.005595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:09.014243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:09.020085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:09.021550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:09.036610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:09.037629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:09.038664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:09.038664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:09.038664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:09.043615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 538/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:09.046751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:09.079151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:09.159312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343259a-8a7e-4aae-a441-378a2db670b4 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.198474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:09.199093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:09.199431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:09.199741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:09.200410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:09.200717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:09.202081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:09.218643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 527/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:09.219818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:09.322748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246156f-f1a4-4fb0-9771-879a73f5344b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.356540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:09.357180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:09.357464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:09.357738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:09.365362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:09.366818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:44:09.378069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:09.379078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:09.380087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 539/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:44:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:09.387231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:09.484632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:09.496298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd34769c-bb35-4947-8830-4600bca2b0ac None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.530101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:09.530768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:09.531049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:09.531316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:09.532070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:09.532374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:09.533787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:09.560791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 528/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:44:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:09.562013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:09.614489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:09.665902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695a203-daf7-4afe-a729-98a615b28a21 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.699572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:09.700222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:09.700499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:09.700768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:09.711179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:09.712807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:09.726861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:09.728056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:09.729002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:09.729002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:09.729002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:09.729002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 540/1068] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:09.735241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:09.834937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b90678-22a1-42c9-b5ee-ead3119f5a3d None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.868996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:09.869626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:09.869903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:09.870170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:09.870804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:09.871126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:09.872512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dd09ec-a16b-49af-bfa2-1a68b6a3e20e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:09.887403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 529/1069] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:09.892373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:09.995978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810722a4-1ccb-4559-b83b-3e1b7bdf2c75 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:09.997419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:10.031448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:10.032107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.032389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.032651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.039198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:10.040671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:44:10.052020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:10.052967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:10.059094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 541/1070] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:44:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:10.059737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d218d8-43ef-4b4a-b142-1e9451e4a031 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.059737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d218d8-43ef-4b4a-b142-1e9451e4a031 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.059737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d218d8-43ef-4b4a-b142-1e9451e4a031 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.059737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 530/1071] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:44:10.061465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:10.062428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.062716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.062993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.083234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:10.084448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:10.086477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:10.156045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 531/1072] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:10.168584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e485bc5-bf1c-4ebf-8b1f-0a95de08ce0f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:10.206048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:10.206659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.206936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.207207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.207922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:10.208232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:10.209652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:10.222277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 542/1073] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:44:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:10.228536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:10.326272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e954e75-23e3-4678-a3f5-814c0026184a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:10.359847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:10.360508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.360782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.361051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/8a9090ab0ffa4b7eafcc742880237229/roles/8f5ed297a5184b2fa9424b2f5429027f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.363164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=8a9090ab0ffa4b7eafcc742880237229, role_id=8f5ed297a5184b2fa9424b2f5429027f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:10.364610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:10.373170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:10.383636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 532/1074] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:44:10 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/8a9090ab0ffa4b7eafcc742880237229/roles/8f5ed297a5184b2fa9424b2f5429027f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:44:10.422673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:10.451741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:10.535064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e27f29-7632-4882-b41d-cd8824ab3c15 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:10.574400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:10.575024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.575303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.575602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.586657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:10.591419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:10.592446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:10.593339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 543/1075] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:10.600308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:10.703026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87530e7e-715a-4751-852b-5f84ffeb49db None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:10.740416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:10.741047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.741327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.741597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.742826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:10.743129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:10.744580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f89bbc-590d-4aa5-8244-252c7f6753b7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:10.758995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 533/1076] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:10.760668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:10.840264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d98c5-2c7a-43e6-a4c6-6503a36a7288 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.840561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d98c5-2c7a-43e6-a4c6-6503a36a7288 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.840745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d98c5-2c7a-43e6-a4c6-6503a36a7288 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.842209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 534/1077] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:10.846360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:10.846925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.847221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.847496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.868505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:10.868724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954ba14-a47a-45b8-95bf-8bbfbf2fbf68 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:10.869703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:10.905041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:10.905691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:10.905973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:10.906245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] PATH_INFO: `/v3/users/dbf5f475aa7346c0a42856e48c0c32ce` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:10.907392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] RBAC: Authorizing `identity:get_user(user_id=dbf5f475aa7346c0a42856e48c0c32ce)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:10.908979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:10.910643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 544/1078] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:10 2022] GET /identity/v3/users/dbf5f475aa7346c0a42856e48c0c32ce => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:10.916658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:10.933415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:10.947584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 535/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:11.018618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def59e6e-301a-4848-ab4a-fb0c164f87f2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:11.052318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:11.052952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:11.053235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:11.053510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:11.055165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] RBAC: Authorizing `identity:get_project(project_id=83b5d6a06ffb49eab4c856a84207e111)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:11.056635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:11.058480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 545/1080] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:10 2022] GET /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:11.064124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:11.167985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb8833-1fda-4256-8b7d-736f76fdae69 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:11.205156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:11.206158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:11.206656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:11.207146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111/users/dbf5f475aa7346c0a42856e48c0c32ce/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:11.209398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] RBAC: Authorizing `identity:create_grant(project_id=83b5d6a06ffb49eab4c856a84207e111, user_id=dbf5f475aa7346c0a42856e48c0c32ce, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:11.211638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:11.225354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:11.226266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 536/1081] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:44:11 2022] PUT /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111/users/dbf5f475aa7346c0a42856e48c0c32ce/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:44:11.249215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:11.342114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4ea8b9-ec8e-4ebd-9828-c84b662eb39b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:11.378671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:11.379307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:11.379617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:11.379897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:11.455873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:11.457251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:11.474888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:11.563580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 546/1082] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 15:44:11.566922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:11.622953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:11.675442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570190c7-b60d-49ae-8812-aab4f6eae9ab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:11.712105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:11.712655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:11.712932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:11.713203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:11.723222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:11.724686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c81172be-578f-4edc-ad1b-47fb20f5916d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:11.739367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:11.740365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:11.741287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:11.741287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:11.741287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:11.745587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 537/1083] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:11.746571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:11.844258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956ab5-6b7d-498f-adcd-11bbeb71c040 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:11.862114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:11.880062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:11.880675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:11.880955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:11.881226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:11.881869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:11.882175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:11.883596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3931114-4ad8-4279-8b54-bd37ef4c681f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:11.903588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 547/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:12.533074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cfc3-64af-4921-a859-c2cb1a8ef482 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:12.533577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cfc3-64af-4921-a859-c2cb1a8ef482 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:12.533577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a103cfc3-64af-4921-a859-c2cb1a8ef482 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:12.534852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 538/1085] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:12.540251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:12.540617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:12.540965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:12.559562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:12.560746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:12.626112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 548/1086] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:12.897503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103cfc3-64af-4921-a859-c2cb1a8ef482 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:12.998862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a103cfc3-64af-4921-a859-c2cb1a8ef482 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.000391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.032945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.033564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.033840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.034111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.044873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:13.049753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:13.050746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:13.051737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 539/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Sep 7 15:44:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:13.057589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.155369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d6c4a-21d1-4d97-b024-89b7777c90c3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.171477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.198177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.198791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.199075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.199350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.200047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:13.200358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:13.201730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb3147-60eb-45a0-a979-f2496941a128 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:13.209624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 549/1088] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:13.216370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.317597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28df86-fa15-44b0-a8db-e5469ee69e95 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.350662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.351306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.351615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.351891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.362393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:13.363848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:13.378102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:13.379142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:13.380117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:13.380117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:13.380117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:13.380117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 540/1089] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:13.380368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41362541-af24-41e5-a43b-b9d5feae8a96 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.380679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41362541-af24-41e5-a43b-b9d5feae8a96 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.380927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41362541-af24-41e5-a43b-b9d5feae8a96 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.382260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 550/1090] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:13.385027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.387215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.387717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.387860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.401222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.405828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:13.406932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:13.472716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 551/1091] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:44:13.486732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf6edd-2303-45e0-880c-8b1217fc7bab None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.518565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.519188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.519463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.519762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.520395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:13.520694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:13.522040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:13.533738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 541/1092] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:13.539384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.633067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.636340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29bd6fd-a325-4c21-98ce-520086a6459a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.669485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.670046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.670323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.670589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.677281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:13.678709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:44:13.690424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:13.691379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:13.692382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 552/1093] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:13.697298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.748413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.770363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.796864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98630b29-4463-4d7b-ab33-8bfc678fe1c7 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.830831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.831439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.831766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.832044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.832767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:13.833076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:13.834493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:13.847011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4adc06-c11b-405f-bbb1-c2b018828059 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.851887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 542/1094] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:13.857810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.880186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.880810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.881093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.881361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:13.892068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:13.896971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:13.897978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:13.898941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 553/1095] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:13.903902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:13.925049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.956752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb81bad-200d-4a25-bbe5-e157f34464c0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:13.988408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:13.993886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:13.994492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:13.994769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:13.995040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.002200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3990aed-aa4d-4805-959e-8d92bd5d6847 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.034348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.034947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.035220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.035487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.036168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:14.036732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.037823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.046520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 554/1096] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:14.052326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.064312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.065764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.133200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.151515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816428d2-17a3-4484-a74f-2ca75ac75380 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.164269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 543/1097] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 07 15:44:14.170108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.187906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.188473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.188748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.189017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] PATH_INFO: `/v3/users/dbf5f475aa7346c0a42856e48c0c32ce` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.190133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] RBAC: Authorizing `identity:get_user(user_id=dbf5f475aa7346c0a42856e48c0c32ce)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.191582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.193156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 555/1098] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/users/dbf5f475aa7346c0a42856e48c0c32ce => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:14.199400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.201709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.269084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a6ecbc-7101-4977-8563-7f72c2837b3a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.295448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97e6625-5e47-401a-88ce-d2afc6bbf2d3 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.302751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.303416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.303732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.304049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.314357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.315829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.327571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.328185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.328470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.328752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:14.330164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:14.331231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:44:14.332198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:14.332198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:44:14.332198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:14.332198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 544/1099] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:14.337407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.347999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.395883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.397332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.424545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.440410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ac1ff3-000e-49dc-9d93-b3b9ef2ebf12 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.473508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.474118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.474396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.474671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.475559 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:14.475682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.477538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.489220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 545/1100] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:14.495285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.498510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 556/1101] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:14.504140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.552125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.597529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6301729f-acf8-4853-aeaf-806d84c30df3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.600908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc5fc7a-4cf8-4914-a4db-28148f76f70c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.632754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.633368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.633644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.633913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.634697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.635326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.635631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.635907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.637066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] RBAC: Authorizing `identity:get_project(project_id=83b5d6a06ffb49eab4c856a84207e111)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.638497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.640178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 557/1102] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:14.640476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.641886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.646135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:44:14.655885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:14.656916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:14.657942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 546/1103] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:44:14.662460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.666671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.748949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41728cd8-a743-4361-89d8-a51b8e464fe1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.765793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:14.766863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820a9c60-33eb-4eb5-a31e-c5c4d0ff000b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.782976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.783601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.783883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.784161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.784873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=83b5d6a06ffb49eab4c856a84207e111, user.id=dbf5f475aa7346c0a42856e48c0c32ce) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:14.785179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.786607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.792388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 558/1104] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/role_assignments?user.id=dbf5f475aa7346c0a42856e48c0c32ce&scope.project.id=83b5d6a06ffb49eab4c856a84207e111&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:14.801091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.801719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.801998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.802270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.802984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:14.803293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.804803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820ea940-d084-48f9-978f-1626b081bef4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.818147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 547/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:14.824703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:14.929764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a508f9-fcc4-4593-8d95-6c89a8477110 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:14.964050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:14.964665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:14.964940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:14.966526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:14.966526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] RBAC: Adding query filter params (role.id=8f5ed297a5184b2fa9424b2f5429027f, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=8a9090ab0ffa4b7eafcc742880237229) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:14.966526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:14.967687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e30b90-9c44-46ae-b90b-67bff5d71128 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:14.973646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 559/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:14 2022] GET /identity/v3/role_assignments?user.id=8a9090ab0ffa4b7eafcc742880237229&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=8f5ed297a5184b2fa9424b2f5429027f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:44:16.951411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d60d-e2bc-4a8f-9d3d-4f0ccbb746b8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:16.951997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d60d-e2bc-4a8f-9d3d-4f0ccbb746b8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:16.951997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3d60d-e2bc-4a8f-9d3d-4f0ccbb746b8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:16.953302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 548/1107] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:16.958533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:16.958946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:16.959094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:16.977361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:16.978555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:17.019731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:17.043803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 560/1108] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:17.308059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3d60d-e2bc-4a8f-9d3d-4f0ccbb746b8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:17.369434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:17.406688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c3d60d-e2bc-4a8f-9d3d-4f0ccbb746b8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:17.438765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ceee6-1f46-451c-8849-f6d273e1c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:17.439369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ceee6-1f46-451c-8849-f6d273e1c283 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:17.439670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ceee6-1f46-451c-8849-f6d273e1c283 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:17.439940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ceee6-1f46-451c-8849-f6d273e1c283 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:17.441017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ceee6-1f46-451c-8849-f6d273e1c283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:17.442422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ceee6-1f46-451c-8849-f6d273e1c283 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:17.443955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 549/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:17.449612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:17.545233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decdf579-8eb6-4670-8ae3-107462761ec1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:17.580329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd51c901-5c68-4d90-bbdb-0f3590c6e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:17.580868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c901-5c68-4d90-bbdb-0f3590c6e16c None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:17.581147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c901-5c68-4d90-bbdb-0f3590c6e16c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:17.581418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd51c901-5c68-4d90-bbdb-0f3590c6e16c None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:17.582064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51c901-5c68-4d90-bbdb-0f3590c6e16c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:17.583487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd51c901-5c68-4d90-bbdb-0f3590c6e16c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:17.605494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 561/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:44:17 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:19.752144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81899caf-bde5-4e21-8d57-40e5b1247d57 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:19.752691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81899caf-bde5-4e21-8d57-40e5b1247d57 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:19.752691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81899caf-bde5-4e21-8d57-40e5b1247d57 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:19.754006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 550/1111] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:19.759809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:19.760194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:19.760350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:19.778614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:19.779821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:19.804314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:19.852652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 562/1112] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:20.119210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81899caf-bde5-4e21-8d57-40e5b1247d57 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:20.219080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81899caf-bde5-4e21-8d57-40e5b1247d57 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:20.247366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:20.252572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:20.253185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:20.253460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:20.253728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] PATH_INFO: `/v3/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:20.255084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] RBAC: Authorizing `identity:get_role(role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:20.256454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:20.258350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 551/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:20 2022] GET /identity/v3/roles/0cc328004b4e4f44905d38cb222157a9 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:20.264111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:20.358513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc584e3-9cc4-40ac-97f3-889e874b6778 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:20.391380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:20.391977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:20.392262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:20.392535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] PATH_INFO: `/v3/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:20.405556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] RBAC: Authorizing `identity:get_user(user_id=5ad80b03ad3b4aa6bc36fa3b7b5367d4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:20.406997 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:20.408580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 563/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:20 2022] GET /identity/v3/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:20.414596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:20.511009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f7ba8-408d-4b19-91fb-1b0c1db61865 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:20.543493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:20.544118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:20.544420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:20.544689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:20.610275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:20.612463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:20.614356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:20.708694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 552/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:20.715204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:20.736624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:20.813575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0ea115-517d-40a8-a921-6a1ad9c3385a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:20.850887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277a2a4-9708-4b0d-8be5-0a860a30e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:20.851521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a2a4-9708-4b0d-8be5-0a860a30e92d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:20.851829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a2a4-9708-4b0d-8be5-0a860a30e92d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:20.852101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a2a4-9708-4b0d-8be5-0a860a30e92d None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:20.853197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277a2a4-9708-4b0d-8be5-0a860a30e92d None admin] RBAC: Authorizing `identity:get_project(project_id=83b5d6a06ffb49eab4c856a84207e111)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:20.854883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f277a2a4-9708-4b0d-8be5-0a860a30e92d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:20.856524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 564/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:20 2022] GET /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:20.862798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:20.955783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:20.958642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ac276b-3508-4f59-9877-e3c11f89d3bf None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:20.991274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:20.992351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:20.992630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:20.992902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:20.993625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] RBAC: Adding query filter params (role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=83b5d6a06ffb49eab4c856a84207e111, user.id=5ad80b03ad3b4aa6bc36fa3b7b5367d4) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:20.993931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:20.995348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63cc362-f66d-4b84-8afe-782a569552b9 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:21.000510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 553/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:20 2022] GET /identity/v3/role_assignments?user.id=5ad80b03ad3b4aa6bc36fa3b7b5367d4&scope.project.id=83b5d6a06ffb49eab4c856a84207e111&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:23.069179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bba05-48e5-4aa3-9c55-b7eae4ff5c43 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:23.069720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bba05-48e5-4aa3-9c55-b7eae4ff5c43 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:23.069835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bba05-48e5-4aa3-9c55-b7eae4ff5c43 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:23.071079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 565/1118] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:23.076119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:23.076537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:23.076670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:23.094818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:23.096049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:23.161133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 554/1119] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:23.430682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bba05-48e5-4aa3-9c55-b7eae4ff5c43 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:23.515969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:23.529016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bba05-48e5-4aa3-9c55-b7eae4ff5c43 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:23.562300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:23.562924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:23.563208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:23.563485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] PATH_INFO: `/v3/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:23.564894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] RBAC: Authorizing `identity:get_role(role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:23.566262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:23.567594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 566/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:23 2022] GET /identity/v3/roles/0cc328004b4e4f44905d38cb222157a9 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:23.573352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:23.664650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:23.671082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34357ad3-a216-4948-a9ed-83c4be96ebe6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:23.705952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:23.706510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:23.706787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:23.707062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] PATH_INFO: `/v3/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:23.708187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] RBAC: Authorizing `identity:get_user(user_id=5ad80b03ad3b4aa6bc36fa3b7b5367d4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:23.709607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:23.711184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 555/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:23 2022] GET /identity/v3/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:23.717000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:23.814552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9649f-6425-4c49-b4ed-a45be3b5f36e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:23.846398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36907420-0183-4f40-988c-cb56514eba74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:23.847015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36907420-0183-4f40-988c-cb56514eba74 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:23.847300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36907420-0183-4f40-988c-cb56514eba74 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:23.847599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36907420-0183-4f40-988c-cb56514eba74 None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:23.848702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36907420-0183-4f40-988c-cb56514eba74 None admin] RBAC: Authorizing `identity:get_project(project_id=83b5d6a06ffb49eab4c856a84207e111)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:23.850123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36907420-0183-4f40-988c-cb56514eba74 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:23.852295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 567/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:23 2022] GET /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:23.858356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:23.954403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ad97-979f-4b4a-a8fb-8e6a6140348c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:23.988148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51ff526-b5d5-4f97-bee7-9874184f054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:23.988730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ff526-b5d5-4f97-bee7-9874184f054d None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:23.989010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ff526-b5d5-4f97-bee7-9874184f054d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:23.989280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51ff526-b5d5-4f97-bee7-9874184f054d None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:23.991430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ff526-b5d5-4f97-bee7-9874184f054d None admin] RBAC: Authorizing `identity:create_grant(project_id=83b5d6a06ffb49eab4c856a84207e111, user_id=5ad80b03ad3b4aa6bc36fa3b7b5367d4, role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:23.993051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51ff526-b5d5-4f97-bee7-9874184f054d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:24.005245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 556/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:44:23 2022] PUT /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4/roles/0cc328004b4e4f44905d38cb222157a9 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:44:26.103672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20285c2-57d6-456d-bc34-e1f2a7de1fca None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:26.104218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20285c2-57d6-456d-bc34-e1f2a7de1fca None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:26.104218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20285c2-57d6-456d-bc34-e1f2a7de1fca None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:26.105803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 568/1124] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:26.111021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:26.111451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:26.111583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:26.129744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:26.130915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:26.147419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:26.197145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 557/1125] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:26.466201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20285c2-57d6-456d-bc34-e1f2a7de1fca None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:26.492261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:26.566526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20285c2-57d6-456d-bc34-e1f2a7de1fca None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:26.598329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:26.598948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:26.599232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:26.599507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] PATH_INFO: `/v3/roles/0cc328004b4e4f44905d38cb222157a9` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:26.601399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] RBAC: Authorizing `identity:get_role(role_id=0cc328004b4e4f44905d38cb222157a9)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:26.602755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:26.604085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 569/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:26 2022] GET /identity/v3/roles/0cc328004b4e4f44905d38cb222157a9 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:26.610175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:26.704166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1307167-4a87-42cf-92f2-195e6f8c3249 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:26.737318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:26.737871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:26.738145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:26.738415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] PATH_INFO: `/v3/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:26.739516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] RBAC: Authorizing `identity:get_user(user_id=5ad80b03ad3b4aa6bc36fa3b7b5367d4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:26.740949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:26.743049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 558/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:26 2022] GET /identity/v3/users/5ad80b03ad3b4aa6bc36fa3b7b5367d4 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:26.748585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:26.836310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:26.845584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29561e20-ae53-44fa-8751-915f41be7762 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:26.880814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:26.881426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:26.881709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:26.882018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:26.952853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:26.954293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:27.038950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:27.048576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 570/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:27.055128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:27.076624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:27.151762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d6df2b-ab35-43d2-a6b9-8c7e37a7f96b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:27.184958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7daee80-8925-42e5-be70-4736f2bc487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:27.185584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7daee80-8925-42e5-be70-4736f2bc487c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:27.185865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7daee80-8925-42e5-be70-4736f2bc487c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:27.186138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7daee80-8925-42e5-be70-4736f2bc487c None admin] PATH_INFO: `/v3/projects/83b5d6a06ffb49eab4c856a84207e111` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:27.187234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7daee80-8925-42e5-be70-4736f2bc487c None admin] RBAC: Authorizing `identity:get_project(project_id=83b5d6a06ffb49eab4c856a84207e111)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:27.188689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7daee80-8925-42e5-be70-4736f2bc487c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:27.190287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 559/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:27 2022] GET /identity/v3/projects/83b5d6a06ffb49eab4c856a84207e111 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:27.197084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:27.291402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72797952-cd8c-40a4-a605-2aa78027180d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:27.323325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:27.324089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:27.324374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:27.324642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:27.325356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] RBAC: Adding query filter params (role.id=0cc328004b4e4f44905d38cb222157a9, scope.project.id=83b5d6a06ffb49eab4c856a84207e111, user.id=5ad80b03ad3b4aa6bc36fa3b7b5367d4) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:27.325662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:27.327071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3a8af5-2b6d-4935-8840-d0f522269194 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:27.332560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 571/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:27 2022] GET /identity/v3/role_assignments?user.id=5ad80b03ad3b4aa6bc36fa3b7b5367d4&scope.project.id=83b5d6a06ffb49eab4c856a84207e111&role.id=0cc328004b4e4f44905d38cb222157a9 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:29.468713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a68ae-eb79-47f1-b899-91635a62f58b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:29.469224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a68ae-eb79-47f1-b899-91635a62f58b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:29.469224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a68ae-eb79-47f1-b899-91635a62f58b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:29.470500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 560/1131] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:29.475748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:29.476140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:29.476300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:29.494557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:29.495759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:29.543676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:29.560981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 572/1132] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:29.833178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a68ae-eb79-47f1-b899-91635a62f58b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:29.919291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:29.932148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a68ae-eb79-47f1-b899-91635a62f58b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:29.964476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ac5fcd-d08f-4f33-a7a9-3471ca7dc5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:29.965079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac5fcd-d08f-4f33-a7a9-3471ca7dc5d6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:29.965354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac5fcd-d08f-4f33-a7a9-3471ca7dc5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:29.965622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ac5fcd-d08f-4f33-a7a9-3471ca7dc5d6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:29.966691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac5fcd-d08f-4f33-a7a9-3471ca7dc5d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:29.968132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ac5fcd-d08f-4f33-a7a9-3471ca7dc5d6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:29.969648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 561/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:29.975527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:30.069644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcee48d3-b079-4f6f-afd2-1b4f7d489d20 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:30.102564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a3cb9-6fb6-4fa9-b2ab-003793653e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:30.103104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3cb9-6fb6-4fa9-b2ab-003793653e99 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:30.103386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3cb9-6fb6-4fa9-b2ab-003793653e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:30.103684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3cb9-6fb6-4fa9-b2ab-003793653e99 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:30.104576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a3cb9-6fb6-4fa9-b2ab-003793653e99 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:30.105995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a3cb9-6fb6-4fa9-b2ab-003793653e99 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:30.131832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 573/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:44:29 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:32.295832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6184a9-d453-4497-8a32-7f40eef81aeb None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:32.296372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6184a9-d453-4497-8a32-7f40eef81aeb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:32.296372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6184a9-d453-4497-8a32-7f40eef81aeb None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:32.297715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 562/1135] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:32.303145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:32.303640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:32.303766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:32.321520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:32.322705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:32.352027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:32.388271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 574/1136] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:32.665738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6184a9-d453-4497-8a32-7f40eef81aeb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:32.763611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6184a9-d453-4497-8a32-7f40eef81aeb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:32.796433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239679f4-29b5-429f-baa4-afa91f3cf062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:32.797045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239679f4-29b5-429f-baa4-afa91f3cf062 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:32.797323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239679f4-29b5-429f-baa4-afa91f3cf062 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:32.797592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239679f4-29b5-429f-baa4-afa91f3cf062 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:32.798684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239679f4-29b5-429f-baa4-afa91f3cf062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:32.800131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239679f4-29b5-429f-baa4-afa91f3cf062 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:32.801651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 563/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:44:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:32.807348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:32.902757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec57ba4-50c6-43e1-a08c-c6be8bdcbf5c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:32.935255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fe0e16-9e04-4f32-99d6-c449a801f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:32.935819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0e16-9e04-4f32-99d6-c449a801f6ea None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:32.936100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0e16-9e04-4f32-99d6-c449a801f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:32.936372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fe0e16-9e04-4f32-99d6-c449a801f6ea None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:32.937017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe0e16-9e04-4f32-99d6-c449a801f6ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:32.938437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fe0e16-9e04-4f32-99d6-c449a801f6ea None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:32.963118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 575/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:44:32 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:35.105354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975a540-f0ae-4085-9730-edd141c12114 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:35.105884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975a540-f0ae-4085-9730-edd141c12114 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:35.105884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2975a540-f0ae-4085-9730-edd141c12114 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:35.107205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 564/1139] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:35.112490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:35.112858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:35.113044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:35.131259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:35.133070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:35.149684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:35.199417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 576/1140] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:35.470871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975a540-f0ae-4085-9730-edd141c12114 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:35.496510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:35.571325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2975a540-f0ae-4085-9730-edd141c12114 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:35.607172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:35.607827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:35.608167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:35.608761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:35.620956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:35.625801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:35.626942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:35.628243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 565/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:35.633488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:35.732914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033934cc-6471-4f1b-8cb8-8bb5c9e50786 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:35.767817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:35.768423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:35.768705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:35.768981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:35.769644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:35.769952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:35.771330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:35.779676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 577/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:35.785600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:35.868261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:35.884449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb170c-d2ee-4586-9fba-1662e84bb4a5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:35.916733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:35.917343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:35.917622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:35.917896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] PATH_INFO: `/v3/users/a927278949ae40e89d5126b012065c28` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:35.931612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] RBAC: Authorizing `identity:get_user(user_id=a927278949ae40e89d5126b012065c28)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:35.933037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:35.934603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 566/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:35 2022] GET /identity/v3/users/a927278949ae40e89d5126b012065c28 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:35.940367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:35.960653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:36.042878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8984b-acf6-49df-bdd2-723f4693628d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:36.075119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:36.075742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:36.076024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:36.076294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:36.139551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:36.141010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:36.156769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:36.235039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 578/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:36.241454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:36.339713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3343f2-11d1-4023-84af-c844e444caad None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:36.373143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdc61e9-25a0-44ba-be07-7679c1b5dd47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:36.373755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc61e9-25a0-44ba-be07-7679c1b5dd47 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:36.374032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc61e9-25a0-44ba-be07-7679c1b5dd47 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:36.374301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc61e9-25a0-44ba-be07-7679c1b5dd47 None admin] PATH_INFO: `/v3/projects/bbdb472587164796976dcc42060285f3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:36.375388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc61e9-25a0-44ba-be07-7679c1b5dd47 None admin] RBAC: Authorizing `identity:get_project(project_id=bbdb472587164796976dcc42060285f3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:36.376835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdc61e9-25a0-44ba-be07-7679c1b5dd47 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:36.378424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 567/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:36 2022] GET /identity/v3/projects/bbdb472587164796976dcc42060285f3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:36.384664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:36.480248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad6a11-8cfb-4a5e-8cce-619ddeb5457f None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:36.496682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:36.514071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261efa65-f221-437f-9e35-866cacce213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:36.514673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261efa65-f221-437f-9e35-866cacce213d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:36.514956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261efa65-f221-437f-9e35-866cacce213d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:36.515230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261efa65-f221-437f-9e35-866cacce213d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:36.515989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261efa65-f221-437f-9e35-866cacce213d None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=bbdb472587164796976dcc42060285f3, user.id=a927278949ae40e89d5126b012065c28) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:36.516305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261efa65-f221-437f-9e35-866cacce213d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:36.517735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261efa65-f221-437f-9e35-866cacce213d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:36.522943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 579/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:36 2022] GET /identity/v3/role_assignments?user.id=a927278949ae40e89d5126b012065c28&scope.project.id=bbdb472587164796976dcc42060285f3&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:38.617725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466d8c-02ae-459a-80bc-056fbd34e1cf None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:38.618243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466d8c-02ae-459a-80bc-056fbd34e1cf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:38.618243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466d8c-02ae-459a-80bc-056fbd34e1cf None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:38.619512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 568/1147] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:38.624782 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:38.625206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:38.625336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:38.643563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:38.644775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:38.710725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 580/1148] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:38.973612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21466d8c-02ae-459a-80bc-056fbd34e1cf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:38.986038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:39.072998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21466d8c-02ae-459a-80bc-056fbd34e1cf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:39.105138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:39.105749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:39.106028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:39.106298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:39.116564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:39.121326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:39.122426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:39.123295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 569/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:39.128487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:39.226204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0052df6a-c259-44e8-8b8f-a58470bdde5b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:39.236494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:39.259636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:39.260234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:39.260508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:39.260774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:39.261418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:39.261717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:39.263073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:39.270809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 581/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:39.277347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:39.345398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:39.373709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f3655-8c73-472a-b244-9f5a9d90cee0 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:39.406172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:39.406786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:39.407064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:39.407337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] PATH_INFO: `/v3/users/a927278949ae40e89d5126b012065c28` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:39.408471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] RBAC: Authorizing `identity:get_user(user_id=a927278949ae40e89d5126b012065c28)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:39.409888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:39.411426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 570/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:39 2022] GET /identity/v3/users/a927278949ae40e89d5126b012065c28 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:39.417745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:39.512948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa02c42-e600-47a9-9d15-b78880a4c668 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:39.544807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382eb50-7fa9-43d4-b2ee-39e077d81053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:39.545427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382eb50-7fa9-43d4-b2ee-39e077d81053 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:39.545709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382eb50-7fa9-43d4-b2ee-39e077d81053 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:39.545981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382eb50-7fa9-43d4-b2ee-39e077d81053 None admin] PATH_INFO: `/v3/projects/bbdb472587164796976dcc42060285f3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:39.547073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382eb50-7fa9-43d4-b2ee-39e077d81053 None admin] RBAC: Authorizing `identity:get_project(project_id=bbdb472587164796976dcc42060285f3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:39.548523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382eb50-7fa9-43d4-b2ee-39e077d81053 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:39.550108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 582/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:39 2022] GET /identity/v3/projects/bbdb472587164796976dcc42060285f3 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:39.556532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:39.652652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8e0b6-7de9-40f4-bf05-b33b14c6faa3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:39.685407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302b8acc-580e-4de4-beee-dd59e2b86a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:39.686045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8acc-580e-4de4-beee-dd59e2b86a64 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:39.686323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8acc-580e-4de4-beee-dd59e2b86a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:39.686598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302b8acc-580e-4de4-beee-dd59e2b86a64 None admin] PATH_INFO: `/v3/projects/bbdb472587164796976dcc42060285f3/users/a927278949ae40e89d5126b012065c28/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:39.688499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b8acc-580e-4de4-beee-dd59e2b86a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=bbdb472587164796976dcc42060285f3, user_id=a927278949ae40e89d5126b012065c28, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:39.689905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302b8acc-580e-4de4-beee-dd59e2b86a64 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:39.698210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:39.705751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 571/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:44:39 2022] PUT /identity/v3/projects/bbdb472587164796976dcc42060285f3/users/a927278949ae40e89d5126b012065c28/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:44:41.819464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947327-e8c4-4fef-8a59-3f53f788d88d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:41.820015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947327-e8c4-4fef-8a59-3f53f788d88d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:41.820015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37947327-e8c4-4fef-8a59-3f53f788d88d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:41.821391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 583/1154] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:41.827476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:41.827873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:41.828059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:41.846586 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:41.847806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:41.912271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 572/1155] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:42.182090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37947327-e8c4-4fef-8a59-3f53f788d88d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:42.224063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:42.281818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37947327-e8c4-4fef-8a59-3f53f788d88d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:42.313754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:42.314374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:42.314657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:42.314932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:42.325225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:42.330034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:42.331108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:42.332257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 584/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:42.337695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:42.433589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b625723-c7f9-425d-834e-627f4011a644 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:42.458690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:42.467330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:42.467951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:42.468226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:42.468493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:42.469145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:42.469448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:42.470792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:42.478484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 573/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:42.484651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:42.579471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:42.582532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0953a-bc00-41cc-a9fe-31ab6889640d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:42.615477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:42.616612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:42.616889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:42.617155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] PATH_INFO: `/v3/users/a927278949ae40e89d5126b012065c28` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:42.618254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] RBAC: Authorizing `identity:get_user(user_id=a927278949ae40e89d5126b012065c28)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:42.619706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:42.621253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 585/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:42 2022] GET /identity/v3/users/a927278949ae40e89d5126b012065c28 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:42.626924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:42.722449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9167ce-59c4-479a-b4f5-621d7a10a4ba None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:42.754493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:42.755075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:42.755351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:42.755659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:42.810438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:42.821300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:42.822730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:42.918197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 574/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:42.924061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:43.019665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937d81e-3724-4908-98f3-6194a305b25c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:43.051300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd05501a-0cf6-424c-a9b7-18e6de06199a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:43.051952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05501a-0cf6-424c-a9b7-18e6de06199a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:43.052235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05501a-0cf6-424c-a9b7-18e6de06199a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:43.052512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd05501a-0cf6-424c-a9b7-18e6de06199a None admin] PATH_INFO: `/v3/projects/bbdb472587164796976dcc42060285f3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:43.054228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05501a-0cf6-424c-a9b7-18e6de06199a None admin] RBAC: Authorizing `identity:get_project(project_id=bbdb472587164796976dcc42060285f3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:43.055687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd05501a-0cf6-424c-a9b7-18e6de06199a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:43.057322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 586/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:42 2022] GET /identity/v3/projects/bbdb472587164796976dcc42060285f3 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:43.063146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:43.149612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:43.159296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a59ad-77b5-4fef-8176-df7ef20c8a66 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:43.193125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:43.193762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:43.194038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:43.194311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:43.195030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=bbdb472587164796976dcc42060285f3, user.id=a927278949ae40e89d5126b012065c28) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:43.195337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:43.196800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a9f25-4762-42c6-a425-5ed0c045728c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:43.202355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 575/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:43 2022] GET /identity/v3/role_assignments?user.id=a927278949ae40e89d5126b012065c28&scope.project.id=bbdb472587164796976dcc42060285f3&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:45.370379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff675d5-4526-4f4b-928d-00d4edbc8e8a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:45.370947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff675d5-4526-4f4b-928d-00d4edbc8e8a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:45.370947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff675d5-4526-4f4b-928d-00d4edbc8e8a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:45.372318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 587/1162] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:45.377553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:45.377998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:45.378098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:45.397853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:45.399046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:45.463595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 576/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:45.728317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff675d5-4526-4f4b-928d-00d4edbc8e8a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:45.765195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:45.827521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff675d5-4526-4f4b-928d-00d4edbc8e8a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:45.859503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:45.860125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:45.860404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:45.860679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:45.923921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:45.925355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:45.957305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:46.019672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 588/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:46.025767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:46.107725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:46.123900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72ce10-c591-4625-b310-dea0333a61a3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:46.156899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:46.157530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:46.158245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:46.158245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:46.165383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:46.166809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-71119063-ba31-4cab-8b3d-cfaa7f771c63 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:46.178105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:46.179379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:44:46.180574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 577/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:44:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:46.185273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:46.280493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c730ba-b668-4b5b-ae3d-9c7dab03d04b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:46.312352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:46.312953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:46.313233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:46.313507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:46.314144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:46.314453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:46.315844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b100a37-f451-40ed-985d-c4e8a7050ef7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:46.318067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:46.324290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 589/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Sep 7 15:44:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:48.419863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23cff0-6443-4ef1-b606-3bb846e32c07 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:48.420389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23cff0-6443-4ef1-b606-3bb846e32c07 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:48.420389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e23cff0-6443-4ef1-b606-3bb846e32c07 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:48.421656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 578/1167] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:48.426750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb58c5d-c175-45e2-9890-6bfea5893e21 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:48.427180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb58c5d-c175-45e2-9890-6bfea5893e21 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:48.427306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb58c5d-c175-45e2-9890-6bfea5893e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:48.445772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2cb58c5d-c175-45e2-9890-6bfea5893e21 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:48.446951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2cb58c5d-c175-45e2-9890-6bfea5893e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:48.511462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 590/1168] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:48.776531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23cff0-6443-4ef1-b606-3bb846e32c07 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:48.874604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e23cff0-6443-4ef1-b606-3bb846e32c07 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:48.906690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e22056-a28b-480a-bf73-fba3dffea241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:48.907277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e22056-a28b-480a-bf73-fba3dffea241 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:48.907563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e22056-a28b-480a-bf73-fba3dffea241 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:48.907850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e22056-a28b-480a-bf73-fba3dffea241 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:48.908496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e22056-a28b-480a-bf73-fba3dffea241 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:48.909837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e22056-a28b-480a-bf73-fba3dffea241 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:48.925520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 579/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Wed Sep 7 15:44:48 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:51.071030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1de9a-41d8-4ae4-ab90-f3f4ca709276 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:51.071620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1de9a-41d8-4ae4-ab90-f3f4ca709276 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:51.071620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb1de9a-41d8-4ae4-ab90-f3f4ca709276 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:51.072969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 591/1170] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:51.078170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:51.078554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:51.078723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:51.086448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:51.097824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:51.099007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:51.167125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 580/1171] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:51.436977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1de9a-41d8-4ae4-ab90-f3f4ca709276 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:51.536876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb1de9a-41d8-4ae4-ab90-f3f4ca709276 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:51.567052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:51.573007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eae25e-80f9-43f5-84e5-a6b8bcf2faac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:51.573628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae25e-80f9-43f5-84e5-a6b8bcf2faac None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:51.573907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae25e-80f9-43f5-84e5-a6b8bcf2faac None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:51.574179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eae25e-80f9-43f5-84e5-a6b8bcf2faac None admin] PATH_INFO: `/v3/domains/dd5ae6eb35b24877b0a4d8eac91d6bcb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:51.584366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eae25e-80f9-43f5-84e5-a6b8bcf2faac None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd5ae6eb35b24877b0a4d8eac91d6bcb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:51.585794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eae25e-80f9-43f5-84e5-a6b8bcf2faac None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:51.587349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 592/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Sep 7 15:44:51 2022] GET /identity/v3/domains/dd5ae6eb35b24877b0a4d8eac91d6bcb => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:51.593271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:51.687973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1feaa8-1c49-4f37-9765-ed408c150e31 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:51.720268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05f2f2-cb72-4ccc-924a-d4b3d985e52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:51.720859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f2f2-cb72-4ccc-924a-d4b3d985e52d None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:51.721140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f2f2-cb72-4ccc-924a-d4b3d985e52d None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:51.721414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05f2f2-cb72-4ccc-924a-d4b3d985e52d None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:51.722050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05f2f2-cb72-4ccc-924a-d4b3d985e52d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:51.723457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec05f2f2-cb72-4ccc-924a-d4b3d985e52d None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:51.728012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:51.750156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 581/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:44:51 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:44:53.875661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497dc6-700f-4c02-b287-a1f5f0a1d1da None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:53.876224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497dc6-700f-4c02-b287-a1f5f0a1d1da None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:53.876224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497dc6-700f-4c02-b287-a1f5f0a1d1da None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:53.877566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 593/1174] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:53.882815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:53.883235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:53.883344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:53.901722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:53.902912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:53.967653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 582/1175] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:54.231619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5497dc6-700f-4c02-b287-a1f5f0a1d1da None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:54.329015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5497dc6-700f-4c02-b287-a1f5f0a1d1da None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:54.361673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f52a0e-163f-406f-890e-550a1efae2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:54.362287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f52a0e-163f-406f-890e-550a1efae2bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:54.362565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f52a0e-163f-406f-890e-550a1efae2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:54.362836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f52a0e-163f-406f-890e-550a1efae2bd None admin] PATH_INFO: `/v3/domains/dd5ae6eb35b24877b0a4d8eac91d6bcb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:54.363952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f52a0e-163f-406f-890e-550a1efae2bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=dd5ae6eb35b24877b0a4d8eac91d6bcb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:54.365390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f52a0e-163f-406f-890e-550a1efae2bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:54.366919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 594/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Sep 7 15:44:54 2022] GET /identity/v3/domains/dd5ae6eb35b24877b0a4d8eac91d6bcb => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:54.372534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:54.466459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51bf21-c3ca-4c70-a9a4-a16ba62b69b4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:54.482953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:54.501047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6a6a5-2eab-4bea-a972-503147018266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:54.501633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6a6a5-2eab-4bea-a972-503147018266 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:54.501909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6a6a5-2eab-4bea-a972-503147018266 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:54.502177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6a6a5-2eab-4bea-a972-503147018266 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:54.502817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6a6a5-2eab-4bea-a972-503147018266 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:54.504254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6a6a5-2eab-4bea-a972-503147018266 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:54.525751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 583/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:44:54 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:44:56.684884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685c9f5-92ba-4a4d-8dfd-003c7595354f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:56.685407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685c9f5-92ba-4a4d-8dfd-003c7595354f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:56.685407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6685c9f5-92ba-4a4d-8dfd-003c7595354f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:56.686797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 595/1178] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:44:56.692116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:56.692517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:56.692661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:56.711416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:44:56.712633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:44:56.781505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 584/1179] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:44:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:44:57.051315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685c9f5-92ba-4a4d-8dfd-003c7595354f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:57.109530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:57.151518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6685c9f5-92ba-4a4d-8dfd-003c7595354f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:57.185128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:57.185760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:57.186045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:57.186323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:57.196805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:44:57.201628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:44:57.202797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:44:57.203741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 596/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:44:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:44:57.208846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:57.304427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b7972-eaa2-45f3-b4d9-d9a4e68e9271 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:57.305846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:57.337299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:57.337896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:57.338175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:57.338446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:57.339105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:57.339426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:57.340797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:57.348484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 585/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:44:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:57.355057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:57.459999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313992ab-5f39-4a8e-b536-2581643e198c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:57.476476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:57.494806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:57.495431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:57.495743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:57.496024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] PATH_INFO: `/v3/users/868967f17a0342f6b877a548b41809e6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:57.509199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] RBAC: Authorizing `identity:get_user(user_id=868967f17a0342f6b877a548b41809e6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:57.510643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:57.512254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 597/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:44:57 2022] GET /identity/v3/users/868967f17a0342f6b877a548b41809e6 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:57.517921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:57.613267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5db50c-00d8-4d25-846a-38ca9ef3bd26 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:57.645253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:57.645839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:57.646114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:57.646893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:57.675893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:57.711056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:57.712517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:57.806111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 586/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:44:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:44:57.812026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:57.907121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a1edc-ed3e-4fe1-bab3-99554d169d8a None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:57.940071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be70961-f91a-4a05-9a34-803c696d07d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:57.940700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70961-f91a-4a05-9a34-803c696d07d9 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:57.940985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70961-f91a-4a05-9a34-803c696d07d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:57.941264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70961-f91a-4a05-9a34-803c696d07d9 None admin] PATH_INFO: `/v3/projects/3c49854560a0463b83719ecb8555f28c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:57.942372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70961-f91a-4a05-9a34-803c696d07d9 None admin] RBAC: Authorizing `identity:get_project(project_id=3c49854560a0463b83719ecb8555f28c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:57.944434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70961-f91a-4a05-9a34-803c696d07d9 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:57.946052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 598/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:44:57 2022] GET /identity/v3/projects/3c49854560a0463b83719ecb8555f28c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:44:57.952060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:44:58.011266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:44:58.046933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b3e96-abc9-4097-a16a-919c43dc7cbe None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:44:58.079152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:44:58.079762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:44:58.080039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:44:58.080304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:44:58.081012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=3c49854560a0463b83719ecb8555f28c, user.id=868967f17a0342f6b877a548b41809e6) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:44:58.081315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:44:58.082714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5758bda8-581e-4545-8da0-0f5c1f6a62d8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:44:58.087842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 587/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:44:57 2022] GET /identity/v3/role_assignments?user.id=868967f17a0342f6b877a548b41809e6&scope.project.id=3c49854560a0463b83719ecb8555f28c&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:00.191100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49a34-ac5d-4dca-b0db-57ebc4151229 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:00.191577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49a34-ac5d-4dca-b0db-57ebc4151229 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:00.191673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd49a34-ac5d-4dca-b0db-57ebc4151229 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:00.193032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 599/1186] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:45:00.198170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:00.198533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:00.198707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:00.217532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:45:00.218701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:45:00.284430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 588/1187] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:45:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:45:00.548400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd49a34-ac5d-4dca-b0db-57ebc4151229 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:00.594147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:00.647395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd49a34-ac5d-4dca-b0db-57ebc4151229 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:00.680499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:00.681114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:00.681390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:00.681656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:00.691268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:45:00.696051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:45:00.697022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:45:00.698010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 600/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:45:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:45:00.703174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:00.759850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:00.800615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138744a-d1dc-4abf-83c6-281078185104 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:00.832234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:00.832820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:00.833095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:00.833362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:00.834526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:45:00.834825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:00.836198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:00.843840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 589/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:00.849486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:00.944176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec033d1-8feb-482b-99dc-e82007a1300e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:00.946386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:00.977161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:00.977770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:00.978048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:00.978323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] PATH_INFO: `/v3/users/868967f17a0342f6b877a548b41809e6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:00.979723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] RBAC: Authorizing `identity:get_user(user_id=868967f17a0342f6b877a548b41809e6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:00.981137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:00.982664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 601/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:45:00 2022] GET /identity/v3/users/868967f17a0342f6b877a548b41809e6 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:00.988202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:01.084523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5147f1-b56f-4133-af16-dc2ab58b9354 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:01.103056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:01.117115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b4092-3893-444e-9017-7a12cb5c6775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:01.117726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4092-3893-444e-9017-7a12cb5c6775 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:01.118007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4092-3893-444e-9017-7a12cb5c6775 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:01.118281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b4092-3893-444e-9017-7a12cb5c6775 None admin] PATH_INFO: `/v3/projects/3c49854560a0463b83719ecb8555f28c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:01.119375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b4092-3893-444e-9017-7a12cb5c6775 None admin] RBAC: Authorizing `identity:get_project(project_id=3c49854560a0463b83719ecb8555f28c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:01.120819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b4092-3893-444e-9017-7a12cb5c6775 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:01.122408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 590/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:45:00 2022] GET /identity/v3/projects/3c49854560a0463b83719ecb8555f28c => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:01.128037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:01.224812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00673c4f-4c18-4e56-bc86-656430135504 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:01.257001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a447b2-b9e6-43fc-bcff-7aa872898a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:01.257647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a447b2-b9e6-43fc-bcff-7aa872898a18 None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:01.257926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a447b2-b9e6-43fc-bcff-7aa872898a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:01.258201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a447b2-b9e6-43fc-bcff-7aa872898a18 None admin] PATH_INFO: `/v3/projects/3c49854560a0463b83719ecb8555f28c/users/868967f17a0342f6b877a548b41809e6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:01.260078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a447b2-b9e6-43fc-bcff-7aa872898a18 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c49854560a0463b83719ecb8555f28c, user_id=868967f17a0342f6b877a548b41809e6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:01.261493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a447b2-b9e6-43fc-bcff-7aa872898a18 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:01.274496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 602/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:45:01 2022] PUT /identity/v3/projects/3c49854560a0463b83719ecb8555f28c/users/868967f17a0342f6b877a548b41809e6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:45:03.362955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cec42-f3f1-474c-af63-03390c343c3d None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:03.363471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cec42-f3f1-474c-af63-03390c343c3d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:03.363614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cec42-f3f1-474c-af63-03390c343c3d None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:03.364864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 591/1193] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:45:03.370436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:03.370833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:03.371000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:03.389306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:45:03.390499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:45:03.415910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:03.456302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 603/1194] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:45:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:45:03.735106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cec42-f3f1-474c-af63-03390c343c3d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:03.834160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cec42-f3f1-474c-af63-03390c343c3d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:03.868725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:03.869343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:03.869626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:03.869901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:03.879767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:45:03.884544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:45:03.885637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:45:03.886931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 592/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:45:03.891684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:03.986530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a8372-ddc3-4305-a11c-3b3de4ba21af None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:04.018585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:04.019179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:04.019460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:04.019755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:04.020407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:45:04.020708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:04.022608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:04.030231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 604/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:04.036102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:04.085068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:04.134784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd903ab4-8888-4b2c-add8-844e5d7db9d8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:04.167978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:04.168606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:04.168883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:04.169152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] PATH_INFO: `/v3/users/868967f17a0342f6b877a548b41809e6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:04.170278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] RBAC: Authorizing `identity:get_user(user_id=868967f17a0342f6b877a548b41809e6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:04.171805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:04.173457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 593/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Sep 7 15:45:04 2022] GET /identity/v3/users/868967f17a0342f6b877a548b41809e6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:04.178982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:04.204310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:04.276887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af13a8-530b-4ae4-8da7-0df36db90d86 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:04.308326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:04.308915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:04.309198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:04.309469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:04.374434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:04.375907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:04.398952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:04.470077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 605/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:45:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:45:04.475984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:04.573292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc068e19-55dc-4942-a40b-f0de4d20932c None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:04.588743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:04.606060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31af5d4-b632-4706-b251-b098ea0dbdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:04.606669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31af5d4-b632-4706-b251-b098ea0dbdf2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:04.606946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31af5d4-b632-4706-b251-b098ea0dbdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:04.607215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31af5d4-b632-4706-b251-b098ea0dbdf2 None admin] PATH_INFO: `/v3/projects/3c49854560a0463b83719ecb8555f28c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:04.608365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31af5d4-b632-4706-b251-b098ea0dbdf2 None admin] RBAC: Authorizing `identity:get_project(project_id=3c49854560a0463b83719ecb8555f28c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:04.609772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31af5d4-b632-4706-b251-b098ea0dbdf2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:04.611342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 594/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Sep 7 15:45:04 2022] GET /identity/v3/projects/3c49854560a0463b83719ecb8555f28c => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:04.617823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:45:04.711842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27bad2e-f2f2-407c-ad8f-47de1abea90e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:45:04.734896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:04.746378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:45:04.746977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:04.747254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:04.747526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:04.748278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=3c49854560a0463b83719ecb8555f28c, user.id=868967f17a0342f6b877a548b41809e6) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:45:04.748585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:45:04.750006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded0c4d-c527-4568-b78e-687b40af8228 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:45:04.756039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 606/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:45:04 2022] GET /identity/v3/role_assignments?user.id=868967f17a0342f6b877a548b41809e6&scope.project.id=3c49854560a0463b83719ecb8555f28c&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:45:14.601902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=8 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:14.753005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:45:59.810282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac592b-1d79-4ed5-8860-692788dc45ce None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:59.810908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac592b-1d79-4ed5-8860-692788dc45ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:59.811521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ac592b-1d79-4ed5-8860-692788dc45ce None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:59.812919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 595/1201] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:45:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:45:59.820008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:45:59.820888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:45:59.821447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:45:59.842580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:45:59.843826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:45:59.930696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 607/1202] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:45:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:00.202873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac592b-1d79-4ed5-8860-692788dc45ce None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:00.329929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ac592b-1d79-4ed5-8860-692788dc45ce None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:00.374312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:00.375416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:00.375864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:00.376363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:00.389581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:00.391380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:00.413839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:00.414963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:00.416647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:00.416647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:00.416647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:00.416647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 596/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:00.421537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:00.556046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d48d80-f905-4d09-bd83-fdffd98fb7eb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:00.592790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:00.601087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:00.601899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:00.602155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:00.602462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:00.603356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:00.603682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:00.605717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413f8a9-a05f-451c-a44c-d6740ba80870 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:00.620821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 608/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:00.626751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:00.699078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:00.762885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81423a51-8356-4d25-8844-f13f5192fa52 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:00.803611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e965e2-e0f5-4eb2-a218-a91f73e7f5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:00.804208 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e965e2-e0f5-4eb2-a218-a91f73e7f5f8 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:00.804475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e965e2-e0f5-4eb2-a218-a91f73e7f5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:00.804765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e965e2-e0f5-4eb2-a218-a91f73e7f5f8 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:00.805382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e965e2-e0f5-4eb2-a218-a91f73e7f5f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:00.806808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e965e2-e0f5-4eb2-a218-a91f73e7f5f8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:00.833175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 597/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Sep 7 15:46:00 2022] POST /identity/v3/users => generated 251 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:46:03.357777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606641-5e4a-432e-8802-63252fe376a4 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:03.358316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606641-5e4a-432e-8802-63252fe376a4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:03.358316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606641-5e4a-432e-8802-63252fe376a4 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:03.359747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 609/1206] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:03.365228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:03.365669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:03.365779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:03.386852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:03.388190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:03.472744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 598/1207] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:03.767214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1606641-5e4a-432e-8802-63252fe376a4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:03.883014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1606641-5e4a-432e-8802-63252fe376a4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:03.922037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:03.922733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:03.923064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:03.923340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:03.937265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:46:03.942748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:03.943819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:03.944705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 610/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:46:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:03.950810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:03.998069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:04.068576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bed13d-417a-4dc3-9a35-546dce5b68dd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:04.113260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:04.113919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:04.114199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:04.114473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:04.115190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:04.115497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:04.116963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:04.130106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 599/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:46:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:04.135972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:04.203919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:04.263718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd50e89-5cbd-43eb-9aec-8277c1c47456 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:04.303391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:04.304158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:04.304483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:04.304802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:04.318796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:04.320828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:04.337890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:04.339270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:04.340439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:04.340439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:04.340439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:04.343771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 611/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:46:04.345390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:04.467132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:04.470463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e37297c-5b40-45f3-9ce0-e27f5b9b4226 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:04.510581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:04.511254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:04.511576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:04.511932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:04.512664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:04.513013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:04.514486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:04.534703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 600/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:04.537016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:04.659106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb89b58-9397-4567-9efc-f58db7dc7572 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:04.660750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:04.697824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:04.698458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:04.698749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:04.699013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:04.706530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:04.708016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:46:04.719886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:04.721009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:04.722073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 612/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:04.727521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:04.840332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f2307e-5c75-40c1-9a7c-695fdd2a06a3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:04.882237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:04.882846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:04.883121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:04.883601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:04.884265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:04.884716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:04.886462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:04.904573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 601/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 15:46:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:04.911019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:05.019184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36fca3-3e50-410c-bae3-be5d06ceaab1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:05.056876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:05.057637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:05.057975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:05.058366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:05.091274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:05.134485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:05.136238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:05.257172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 613/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:46:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:46:05.263226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:05.283315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:05.374572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebbf583-c7a9-4652-b6fa-a0373c494413 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:05.412412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:05.413053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:05.413395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:05.413712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:05.424591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:05.426627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:05.443533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:05.444677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:05.446155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:05.446155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:05.446155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:05.447707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 602/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:05.450357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:05.516835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:05.561839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b300a6a-62b7-4d9c-b390-3cceb09ee15e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:05.604469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:05.605084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:05.605391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:05.605724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:05.606294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:05.606608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:05.608081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:05.625411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 614/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:05.628638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:05.712725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:05.759644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c719c5-8f30-402e-a3d5-0e1cc447fe04 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:05.805982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:05.806869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:05.807153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:05.807507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:05.816700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:05.818129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:46:05.833880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:05.834972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:05.841078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 603/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:46:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:05.841623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:05.954174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89bb7a3-d2b3-42d4-84e9-6d072d914a2b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:05.984324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:05.998236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:05.998825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:05.999107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:05.999416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:06.000198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:06.000512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:06.001957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b1d18-b98e-4411-9de1-16b3d5a4b616 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:06.014244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 615/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:46:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:06.021270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:06.134352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31813959-1b45-4361-86e0-f7ff4f637f24 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:06.166674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:06.174857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:06.176129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:06.176607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:06.176915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:06.177647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=cc95332f41224ecdb42604ac98262545) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:06.178247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:06.179898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7da83-2505-4741-ae93-e30a1b953ce0 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:06.187962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 604/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:46:06 2022] GET /identity/v3/role_assignments?user.id=cc95332f41224ecdb42604ac98262545&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:08.448544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1ec18-3358-40b6-b06d-f1923e477a3d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:08.449102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1ec18-3358-40b6-b06d-f1923e477a3d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:08.449102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1ec18-3358-40b6-b06d-f1923e477a3d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:08.450352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 616/1220] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:08.455984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:08.456942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:08.457152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:08.476398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:08.477818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:08.557925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 605/1221] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:08.857081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1ec18-3358-40b6-b06d-f1923e477a3d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:08.970821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1ec18-3358-40b6-b06d-f1923e477a3d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:09.011064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:09.011825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:09.012182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:09.012530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:09.023137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:46:09.028470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:09.029823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:46:09.030963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:09.038553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 617/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:46:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:09.038750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:09.152340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034632e2-c98d-41ef-8e23-603e074b24ff None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:09.188835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:09.196719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:09.197499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:09.197878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:09.198324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:09.199091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] RBAC: Adding query filter params (name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:09.199531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:09.201071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:09.210681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 606/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:46:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:46:09.216412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:09.267907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:09.340267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44565fd-8864-48af-b6a1-c1eff57f0800 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:09.380033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:09.381002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:09.381524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:09.381837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:09.394836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:09.396557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:09.418234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:09.419470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:09.420616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:09.420616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:09.420616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:09.420616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 618/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:09.426254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:09.534002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebad2b7-3703-49aa-a9e5-345e38753b72 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:09.573358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:09.574132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:09.574475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:09.574901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:09.575805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:09.576178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:09.577858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:09.593140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 607/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:09.600630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:09.708786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:09.716602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ead1b-42d4-49da-82ce-048d7eda5bc0 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:09.754331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:09.755228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:09.755507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:09.755822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:09.763487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:09.765006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:46:09.777524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:09.779517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:09.781876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 619/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:46:09.785850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:09.838341 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:09.896692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd99d41-aaa0-4b39-9f7d-22e02d3a1f9b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:09.934551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:09.935156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:09.935502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:09.935807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:09.936630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:09.936936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:09.938477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:09.957653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 608/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 15:46:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:46:09.963568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:10.072567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da190fe-09b7-43e4-9a3b-8c96df0c1aaa None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:10.108064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:10.108732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:10.109017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:10.109291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:10.121263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:10.122879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:10.138737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:10.139840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:10.141378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:10.141378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:10.141378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:10.141378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 620/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:46:10.146955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:10.265074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921db53-4888-4aad-94b8-7101caff2327 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:10.267537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:10.305278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:10.305899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:10.306175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:10.306444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:10.307085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:10.307388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:10.308780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:10.320520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 609/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:10.327439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:10.336764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:10.437109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8b373-56e9-44ea-82ce-15958e0a9636 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:10.475983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:10.476623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:10.476906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:10.477180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:10.484060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:10.485771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:46:10.498892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:10.500046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:10.501176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 621/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:46:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:10.506753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:10.624723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fad752-3545-45d4-b76b-32047f4a2be3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:10.666294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:10.666912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:10.667192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:10.667463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:10.668198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:10.668498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:10.669971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745bd6-09e4-4c64-82f8-72ccd792f3fc None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:10.682878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 610/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:46:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:10.689028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:10.757742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:10.797028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c7bc7-4810-431e-9503-5c9a1f9c83bd None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:10.835113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e59d4d-c194-4c8e-aecf-65f0890dddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:10.836154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e59d4d-c194-4c8e-aecf-65f0890dddfe None admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:10.836651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e59d4d-c194-4c8e-aecf-65f0890dddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:10.836970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e59d4d-c194-4c8e-aecf-65f0890dddfe None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/cc95332f41224ecdb42604ac98262545/roles/407d62836ce64e0b870a6cb5556902c1` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:10.851912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e59d4d-c194-4c8e-aecf-65f0890dddfe None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=cc95332f41224ecdb42604ac98262545, role_id=407d62836ce64e0b870a6cb5556902c1)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:10.853517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e59d4d-c194-4c8e-aecf-65f0890dddfe None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:10.866167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 622/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:46:10 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/cc95332f41224ecdb42604ac98262545/roles/407d62836ce64e0b870a6cb5556902c1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:46:13.168247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60539f-0d7e-4dd0-81d4-8c998496a2a5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:13.168689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60539f-0d7e-4dd0-81d4-8c998496a2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:13.168829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60539f-0d7e-4dd0-81d4-8c998496a2a5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:13.170486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 611/1233] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:13.176075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:13.176565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:13.176565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:13.196335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:13.197531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:13.272423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 623/1234] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:13.554461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60539f-0d7e-4dd0-81d4-8c998496a2a5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:13.584556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:13.666924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60539f-0d7e-4dd0-81d4-8c998496a2a5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:13.702770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:13.703396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:13.703715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:13.703986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:13.714281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:46:13.719095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:13.723631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:13.724795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 612/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Sep 7 15:46:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:13.726519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:13.773936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:13.832312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1950b40d-fd31-45c9-a64d-2ac462f8b5a2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:13.865870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:13.866479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:13.866764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:13.867042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:13.867741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] RBAC: Adding query filter params (name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:13.868051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:13.869423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:13.877159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 624/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Sep 7 15:46:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:13.883455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:13.974096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:13.992147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f624ed-2afe-4f2d-ae5b-891ba2178c8f None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:14.030907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:14.031467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:14.031781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:14.032073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:14.042551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:14.044192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:14.060676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:14.061802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:14.062727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:14.062727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:14.062727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:14.062727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 613/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:14.068458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:14.182316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1b082-7061-4763-9344-5d7f2514f46c None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:14.192611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:14.219644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:14.220289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:14.220574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:14.220851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:14.221496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:14.222367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:14.223765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:14.236100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 625/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:14.241977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:14.349674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef55c6f5-8bb5-4249-83a3-14027348f083 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:14.387821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:14.388445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:14.388718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:14.389010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:14.392030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:14.398987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:14.400653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:46:14.414159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:14.415343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:14.416763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 614/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:14.421475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:14.530066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecd4afb-0eb5-4bae-8862-f4f8391e10b5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:14.572878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:14.573526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:14.573813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:14.574087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:14.574912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:14.575139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:14.576662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:14.596148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 626/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 15:46:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:14.602879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:14.719604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ac277-a7d9-46b8-b5ac-0a7f643f7777 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:14.759047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:14.759732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:14.760049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:14.760295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:14.833537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:14.841387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:14.842940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:14.956290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 615/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:46:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:46:14.962354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:14.996156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:15.081474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdf08c-47e8-4e37-86cb-aa8d8b82cca2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:15.119192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:15.119967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:15.120380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:15.120665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:15.132528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:15.133987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:15.151124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:15.152433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:15.153381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:15.153381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:15.153381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:15.153381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 627/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:46:15.158758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:15.260843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:15.267131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d68d8-94a1-4b5b-a1c3-1f13eaeca242 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:15.306656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:15.307281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:15.307590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:15.307870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:15.308519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:15.308822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:15.310188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:15.322047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 616/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:15.327844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:15.416214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:15.441157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29124ddc-5ca7-4002-9e6b-fb390c4fbb70 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:15.478282 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:15.479205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:15.479583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:15.479896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:15.486672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:15.488528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:46:15.501454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:15.502569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:15.504316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 628/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:46:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 07 15:46:15.509705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:15.635362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa4e54-1ef6-4683-9e33-ddff224551be None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:15.674880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:15.675748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:15.676115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:15.676402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:15.677240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:15.677525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:15.679477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e967ad2c-190f-4e6f-b7ee-1bd8dbad4187 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:15.693778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 617/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:46:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:46:15.700663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:15.810589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fe549f-b86a-4844-94bf-a63f11fd1575 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:15.847163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:15.847818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:15.848109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:15.848476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:15.849257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] RBAC: Adding query filter params (role.id=407d62836ce64e0b870a6cb5556902c1, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=cc95332f41224ecdb42604ac98262545) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:15.849642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:15.851129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ce57e-d7b5-4634-8ab4-bdaf97e15167 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:15.857911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 629/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:46:15 2022] GET /identity/v3/role_assignments?user.id=cc95332f41224ecdb42604ac98262545&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=407d62836ce64e0b870a6cb5556902c1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:18.092977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e4651-b754-45cd-ab86-ce144d31f62f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:18.093637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e4651-b754-45cd-ab86-ce144d31f62f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:18.093637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1e4651-b754-45cd-ab86-ce144d31f62f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:18.095223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 618/1247] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:18.100753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:18.101200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:18.101324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:18.108843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:18.123203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:18.124482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:18.207404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 630/1248] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:18.480911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e4651-b754-45cd-ab86-ce144d31f62f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:18.500780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:18.606793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1e4651-b754-45cd-ab86-ce144d31f62f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:18.645348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:18.645995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:18.646268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:18.646539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:18.657214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:46:18.662605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:18.668751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:18.669913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 619/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:18.670887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:18.777913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc0022-c6c8-42e9-a55c-ecd850cb8c08 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:18.808625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:18.814498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:18.815170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:18.815444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:18.815708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:18.816375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:18.816684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:18.818055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:18.826275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 631/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:46:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:18.832870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:18.920180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:18.946289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f3191-336e-4578-a0db-5c80a9d2656a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:18.985024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:18.985655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:18.985933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:18.986331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:18.998145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:18.999661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:19.015858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:19.017132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:19.018272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:19.018272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:19.018272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:19.018272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 620/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:19.023207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:19.129571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8704d9ae-54d8-4fc2-a945-0029532c25c4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:19.164946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:19.165574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:19.165853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:19.166124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:19.166781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:19.167087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:19.168517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:19.189786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 632/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:19.191029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:19.296286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13dc2a6-2b5d-4e19-a47d-6b7f34503a8b None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:19.325178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:19.333520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:19.334157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:19.334440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:19.334709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:19.343107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:19.344614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:46:19.357704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:19.358951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:19.360146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 621/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:19.365641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:19.424024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:19.468037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b4765-443f-4d6b-9526-847c8c9f2b87 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:19.503013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:19.503690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:19.503975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:19.504254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:19.504988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:19.505295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:19.506733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:19.524624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 633/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 15:46:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:19.530590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:19.654490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55d980-56cd-4a75-882b-93d2ff408394 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:19.687076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:19.687637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:19.687917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:19.688188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:19.741019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:19.753885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:19.755304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:19.848631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 622/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:46:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:46:19.854702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:19.951408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94040ba-5620-4784-a68f-c835700cdf79 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:19.963991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:19.985314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:19.985940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:19.986223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:19.986498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:19.996184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:19.997637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:20.014861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:20.015966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:20.016907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:20.016907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:20.016907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:20.016907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 634/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:20.022359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:20.106182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:20.118542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d29416-08e7-4483-905f-adeeff84143a None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:20.152985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:20.153585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:20.153861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:20.154129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:20.154759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:20.155057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:20.156450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:20.167606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 623/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:20.173146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:20.271617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5323a013-5768-4c36-8326-4110bd6530f5 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:20.305483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:20.306100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:20.306378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:20.306649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:20.315162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:20.317433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:46:20.328404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:20.329498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:20.330816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 635/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:46:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:20.335304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:20.430848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0fc1e9-d91d-4712-827d-0f053ec5baa2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:20.463063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:20.463677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:20.463954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:20.464226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:20.464902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:20.465201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:20.466597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f76d7-46ac-4023-8fb8-aede6541750e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:20.478105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 624/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:46:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:20.484320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:20.510139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:20.585472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec489a0c-7ba7-458b-95c0-f982dccef94d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:20.622657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:20.623265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:20.623584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:20.623873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:20.624599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=cc95332f41224ecdb42604ac98262545) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:20.624917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:20.626367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502a72d0-1d2a-48fe-80b8-88e2766b330c None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:20.631816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 636/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:46:20 2022] GET /identity/v3/role_assignments?user.id=cc95332f41224ecdb42604ac98262545&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:22.717931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3618bcc-fa03-4487-b90b-ea500772e7ba None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:22.718472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3618bcc-fa03-4487-b90b-ea500772e7ba None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:22.718472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3618bcc-fa03-4487-b90b-ea500772e7ba None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:22.719885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 625/1261] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:22.724955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:22.725333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:22.725509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:22.744597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:22.745765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:22.809246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 637/1262] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:23.069993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3618bcc-fa03-4487-b90b-ea500772e7ba None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:23.080806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:23.169284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3618bcc-fa03-4487-b90b-ea500772e7ba None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:23.201597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:23.202217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:23.202495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:23.202767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:23.213045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:46:23.217875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:23.218962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:23.220072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 626/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:46:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:23.225631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:23.243475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:23.319621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce714e6-a649-4666-b711-a00dfc48b965 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:23.352409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:23.352947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:23.353223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:23.353491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:23.354402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:23.354703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:23.356091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:23.365311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 638/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:46:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:23.371139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:23.437917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:23.469970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21994ed7-a4d5-4b8f-93b0-d655864a0689 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:23.503280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:23.503920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:23.504198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:23.504467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:23.514076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:23.515507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:23.530090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:23.531200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:23.532325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:23.532325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:23.532325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:23.532325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 627/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:23.536886 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:23.635060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:23.654537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b8be4-04c8-4d86-b831-306f62e5fffb None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:23.691128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:23.691752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:23.692032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:23.692302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:23.692934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:23.693239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:23.694599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:23.706341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 639/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:23.712048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:23.807933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387d4b2-c817-4845-a34b-646d35e8d5b6 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:23.826842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:23.840527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:23.841134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:23.841410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:23.841676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:23.849335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:23.850736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:46:23.862826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:23.863831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:23.864937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 628/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:23.869631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:23.964042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ded550-9154-4362-901e-4e8b8147f5d4 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:24.000284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:24.000883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:24.001162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:24.001432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:24.002154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:24.002486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:24.003940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:24.012334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:24.021475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 640/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 15:46:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:24.027281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:24.122814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb547b11-1388-4556-8781-50129be7e1c8 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:24.154645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:24.155253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:24.155526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:24.155821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:24.166325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:24.167772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:24.181982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:24.183144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:24.184457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:24.184457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:24.184457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:24.184457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 629/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:24.188642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:24.289424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e9c12-8909-478a-a330-b5b6cd8d4aa1 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:24.322630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:24.323240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:24.323523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:24.323961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:24.324465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:24.324769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:24.326129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:24.336955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 641/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:24.342743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:24.375248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:24.440698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7288d-73b2-45f9-8d48-596cadf8eba5 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:24.472694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:24.473301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:24.473577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:24.473846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:24.480473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:24.481888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:46:24.493329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:24.494414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:24.495605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 630/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:24.500315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:24.557491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:24.606320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef05f09-5fa4-4aa7-9759-6984cd9bff69 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:24.640124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:24.640727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:24.641007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:24.641280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:24.641966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:24.642272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:24.643745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd93b998-ea15-4632-b8e3-65e370c3a604 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:24.655180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 642/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:46:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:24.661160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:24.749411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:24.758878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e5d5f1-0b92-4b8d-9258-6455975a7895 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:24.790733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995494a-d8d4-4120-aeff-2f85c69edb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:24.791376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995494a-d8d4-4120-aeff-2f85c69edb29 None admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:24.791685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995494a-d8d4-4120-aeff-2f85c69edb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:24.791960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995494a-d8d4-4120-aeff-2f85c69edb29 None admin] PATH_INFO: `/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/cc95332f41224ecdb42604ac98262545/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:24.793816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995494a-d8d4-4120-aeff-2f85c69edb29 None admin] RBAC: Authorizing `identity:create_grant(project_id=d73a0e7260464132bcc1491e2ebb347d, user_id=cc95332f41224ecdb42604ac98262545, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:24.795239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995494a-d8d4-4120-aeff-2f85c69edb29 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:24.808729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 631/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Sep 7 15:46:24 2022] PUT /identity/v3/projects/d73a0e7260464132bcc1491e2ebb347d/users/cc95332f41224ecdb42604ac98262545/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:46:26.922929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07ebf-490a-47cb-a453-e329fa677b44 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:26.923482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07ebf-490a-47cb-a453-e329fa677b44 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:26.923482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce07ebf-490a-47cb-a453-e329fa677b44 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:26.925345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 643/1274] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:26.930501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:26.930930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:26.931025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:26.949619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:26.950808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:27.016318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 632/1275] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:27.287558 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07ebf-490a-47cb-a453-e329fa677b44 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:27.384967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce07ebf-490a-47cb-a453-e329fa677b44 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:27.401253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:27.418120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:27.418735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:27.419013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:27.419283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:27.429501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 07 15:46:27.434209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:27.435318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:27.436279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 644/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Sep 7 15:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:27.441342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:27.519478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:27.539449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfedbc88-454c-45e0-9d1e-0dfd8061cc3e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:27.577465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:27.578090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:27.578445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:27.578788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:27.579601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:27.580054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:27.581732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:27.597398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 633/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Sep 7 15:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:27.598892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:27.701352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05a12e8-f5ec-458c-92c3-bdc305d9fe0e None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:27.734630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:27.735252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:27.735534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:27.735836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:27.745462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:27.746912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:27.761105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:27.762174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:27.763302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:27.763302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:27.763302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:27.763302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 645/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:27.768570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:27.865419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefec583-3df2-495c-868b-e49f79657368 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:27.897736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:27.898324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:27.898602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:27.898871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:27.899499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:27.899828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:27.901168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:27.912951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 634/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:27.918418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:27.947642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:28.016073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958916bb-7f6c-46bf-b1ec-2a58597b12c7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:28.048249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:28.048872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:28.049178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:28.049452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:28.056500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:28.057950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 07 15:46:28.069385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:28.070491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:28.071491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 646/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:28.076268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:28.080071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:28.175329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c7d1d-28d6-4998-b460-21851dcf704e None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:28.210026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:28.210622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:28.210896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:28.211165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:28.211906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:28.212210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:28.213623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:28.230198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 635/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Sep 7 15:46:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:28.235885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:28.323291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:28.333920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c1b6a-bc6c-4111-a7bd-bd9c39b186bc None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:28.366198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:28.366792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:28.367151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:28.367429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:28.433551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:28.435068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:28.522218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:28.532819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 647/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Sep 7 15:46:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 07 15:46:28.538560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:28.600301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:28.637916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662a283-ad8a-474b-995a-f77434e6cf09 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:28.669815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:28.670430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:28.670710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:28.670982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:28.681092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:28.682508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 07 15:46:28.696620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:28.697744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 07 15:46:28.698721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 07 15:46:28.698721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 07 15:46:28.698721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:28.698721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 636/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:46:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:28.703260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:28.797278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b1535e-852a-4370-afed-eb550d9d657b None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:28.829338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:28.829928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:28.830205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:28.830536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:28.831163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:28.831463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:28.832846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:28.843434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 648/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Sep 7 15:46:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:28.849308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:28.945035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105cdb6-3ac4-425e-b934-2818eb0b1dd2 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:28.957876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:28.977720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:28.978321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:28.978850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:28.979120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:28.985403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:28.987397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 07 15:46:28.998200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:28.999226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: ERROR keystone.server.flask.application Sep 07 15:46:29.000533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 637/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Sep 7 15:46:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:29.004925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:29.054435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:29.103305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588dec8-4af6-49c5-ab8e-e477259eaee2 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:29.135187 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:29.135813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:29.136096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:29.136371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:29.137057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:29.137362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:29.138790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd778b8d-52a8-483f-8bf0-3503525f0f05 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:29.150709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 649/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Sep 7 15:46:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:29.156739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:29.252885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cd808-3c4a-4980-b042-e51bd42a3807 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:29.285140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:29.285723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:29.286000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:29.286270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:29.286979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] RBAC: Adding query filter params (role.id=047a84bbadcb4ccf9187faec9f7853fc, scope.project.id=d73a0e7260464132bcc1491e2ebb347d, user.id=cc95332f41224ecdb42604ac98262545) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:29.287283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:29.288723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0870c6-fa6e-4dc3-8e7c-3dd6a217e9e4 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:29.294196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 638/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Sep 7 15:46:29 2022] GET /identity/v3/role_assignments?user.id=cc95332f41224ecdb42604ac98262545&scope.project.id=d73a0e7260464132bcc1491e2ebb347d&role.id=047a84bbadcb4ccf9187faec9f7853fc => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:31.405175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80092-8893-465a-85b6-fe59d47932b0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:31.405700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80092-8893-465a-85b6-fe59d47932b0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:31.405700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e80092-8893-465a-85b6-fe59d47932b0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:31.407048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 650/1288] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:31.412179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:31.412562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:31.412653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:31.430959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:31.432353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:31.464634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:31.501281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 639/1289] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:31.792223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e80092-8893-465a-85b6-fe59d47932b0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:31.888967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e80092-8893-465a-85b6-fe59d47932b0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:31.899017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:31.921906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:31.922522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:31.922801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:31.923070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:31.923665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:31.925586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:31.934051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:31.935274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:31.936396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 651/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:46:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:31.941034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:32.036476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ebd415-cc42-4349-b40d-2b7751ae8ac7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:32.069439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:32.069972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:32.070250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:32.070521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:32.071144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:32.071453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:32.072867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e1256a-2501-4107-b5f1-06ebfa6dfeb6 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:32.077932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 640/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 15:46:31 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:32.083469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:32.180431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283cf32f-806e-4c9e-a16f-d3d398cab84d None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:32.212439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:32.213349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:32.213629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:32.213899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] PATH_INFO: `/v3/services` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:32.214517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:32.214822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:32.216201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda0a29-b109-49ee-94b8-9a5925eedb34 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:32.220449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 652/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 15:46:32 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:34.319323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1138-9e99-4fe3-9d07-a53b26c9978b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:34.319867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1138-9e99-4fe3-9d07-a53b26c9978b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:34.319978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531c1138-9e99-4fe3-9d07-a53b26c9978b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:34.321338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 641/1293] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:34.326628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675aa46-5927-4325-8888-654e7be74074 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:34.327023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675aa46-5927-4325-8888-654e7be74074 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:34.327181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2675aa46-5927-4325-8888-654e7be74074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:34.345273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2675aa46-5927-4325-8888-654e7be74074 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:34.346477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2675aa46-5927-4325-8888-654e7be74074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:34.374948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:34.411190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 653/1294] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:34.697469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1138-9e99-4fe3-9d07-a53b26c9978b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:34.730094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:34.797964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531c1138-9e99-4fe3-9d07-a53b26c9978b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:34.830433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cbdc3-b4ee-4a8f-8de4-98994b5b8379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:34.831020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cbdc3-b4ee-4a8f-8de4-98994b5b8379 None admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:34.831298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cbdc3-b4ee-4a8f-8de4-98994b5b8379 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:34.831588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cbdc3-b4ee-4a8f-8de4-98994b5b8379 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:34.832231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cbdc3-b4ee-4a8f-8de4-98994b5b8379 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:34.833578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cbdc3-b4ee-4a8f-8de4-98994b5b8379 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:34.847800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 642/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Sep 7 15:46:34 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:46:36.986010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dc0db-858e-43f3-9fcd-3c808e9a06ff None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:36.987075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dc0db-858e-43f3-9fcd-3c808e9a06ff None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:36.987075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74dc0db-858e-43f3-9fcd-3c808e9a06ff None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:36.988420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 654/1296] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:36.993580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:36.993980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:36.994114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:37.012369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:37.013543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:37.082744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 643/1297] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 07 15:46:37.350022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dc0db-858e-43f3-9fcd-3c808e9a06ff None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:37.448691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74dc0db-858e-43f3-9fcd-3c808e9a06ff None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:37.481562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:37.482184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:37.482466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:37.482742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:37.483323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:37.484724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:37.493824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:37.494897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:37.496092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 655/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:46:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:37.500720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:37.518892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:37.600380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304689a-368b-404f-9c57-efaf3dfad303 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:37.634015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:37.634614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:37.634895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:37.635166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:37.635821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:37.636130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:37.637491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:37.642409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 644/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 15:46:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:46:37.648469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:37.679396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:37.746719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6143a37-fa15-4bbe-aba3-078a872bb094 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:37.779331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:37.779954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:37.780236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:37.780510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:37.781210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] RBAC: Adding query filter params (interface=public, service_id=31750b85b6134ccdb5f419e932c20537, region_id=RegionOne) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:37.781523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:37.782883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738c91f-5951-4b05-b997-357a95cc9dec None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:37.788341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 656/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Sep 7 15:46:37 2022] GET /identity/v3/endpoints?service_id=31750b85b6134ccdb5f419e932c20537&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:37.793838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:37.872760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:37.890823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21e8841-09d0-453b-b9f6-9dac418585f3 None admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:37.923313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dd3e5a-ced5-4541-81ba-f99d833bafdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:37.924481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3e5a-ced5-4541-81ba-f99d833bafdd None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:37.924757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3e5a-ced5-4541-81ba-f99d833bafdd None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:37.925023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dd3e5a-ced5-4541-81ba-f99d833bafdd None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:37.925624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd3e5a-ced5-4541-81ba-f99d833bafdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:37.926966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dd3e5a-ced5-4541-81ba-f99d833bafdd None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:37.933541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 645/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Sep 7 15:46:37 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:46:40.062177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef63b58-c3b3-4f07-bb42-5bca15bf03b9 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:40.062705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef63b58-c3b3-4f07-bb42-5bca15bf03b9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:40.062794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef63b58-c3b3-4f07-bb42-5bca15bf03b9 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:40.064108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 657/1302] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:46:40.069846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:40.070234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:40.070405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:40.088777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:46:40.089951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:46:40.153625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 646/1303] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:46:40 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 07 15:46:40.421824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef63b58-c3b3-4f07-bb42-5bca15bf03b9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:40.512854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:40.520197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef63b58-c3b3-4f07-bb42-5bca15bf03b9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:40.552967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:40.553586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:40.553870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:40.554140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:40.554712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:40.556099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 07 15:46:40.564624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 07 15:46:40.565739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.get_or_create( Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application with Lock( Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self._enter() Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application return self.creator() Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application created_value = creator( Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: ERROR keystone.server.flask.application Sep 07 15:46:40.566626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 658/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Sep 7 15:46:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 07 15:46:40.571588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:40.645705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:40.668347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4624ea-5a0e-4337-8f57-9baa2151bb79 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:40.700227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:40.700769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:40.701045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:40.701315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] PATH_INFO: `/v3/services` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:40.701939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:46:40.702239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:40.703601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cadd6f-87ad-480e-a2e5-4796e42e3119 None admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:40.709020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 647/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Sep 7 15:46:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:46:40.714698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:46:40.810489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e924f8-d456-488e-81fc-ff3f58c87af7 None admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:46:40.842689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c75757-a5b9-4d49-9eab-40b2bdda1c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:46:40.843270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c75757-a5b9-4d49-9eab-40b2bdda1c06 None admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:46:40.843568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c75757-a5b9-4d49-9eab-40b2bdda1c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:46:40.843843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c75757-a5b9-4d49-9eab-40b2bdda1c06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:46:40.844492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c75757-a5b9-4d49-9eab-40b2bdda1c06 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:46:40.845871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c75757-a5b9-4d49-9eab-40b2bdda1c06 None admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:46:40.864724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 659/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:46:40 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:46:50.706188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=30 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:46:50.860325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:44.444892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc57615-f173-442b-9364-f625753a251c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:44.445317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc57615-f173-442b-9364-f625753a251c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:44.445397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc57615-f173-442b-9364-f625753a251c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:44.452170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316083a-fda2-48a4-ba45-e17a235979b3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:44.453116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316083a-fda2-48a4-ba45-e17a235979b3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:44.453247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316083a-fda2-48a4-ba45-e17a235979b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:44.457070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 648/1307] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:47:44 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:47:44.589623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5316083a-fda2-48a4-ba45-e17a235979b3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:47:44.591086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5316083a-fda2-48a4-ba45-e17a235979b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:44.786470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 660/1308] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:47:45.087441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf89e1e-8079-4236-8a3b-cf8d04984e8c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:45.087835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf89e1e-8079-4236-8a3b-cf8d04984e8c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:45.088017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf89e1e-8079-4236-8a3b-cf8d04984e8c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:45.089665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 649/1309] 10.222.0.17 () {54 vars in 965 bytes} [Wed Sep 7 15:47:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:47:45.096242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9f352-b9bf-4813-af7c-b10d512926cc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:45.096563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9f352-b9bf-4813-af7c-b10d512926cc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:45.096797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9f352-b9bf-4813-af7c-b10d512926cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:45.198207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-00b9f352-b9bf-4813-af7c-b10d512926cc None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:47:45.199650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-00b9f352-b9bf-4813-af7c-b10d512926cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:45.224315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:45.305445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf89e1e-8079-4236-8a3b-cf8d04984e8c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:47:45.307299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-daf89e1e-8079-4236-8a3b-cf8d04984e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:45.312045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 661/1310] 10.222.0.17 () {58 vars in 1062 bytes} [Wed Sep 7 15:47:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:47:45.443470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf89e1e-8079-4236-8a3b-cf8d04984e8c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:47:45.527298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:47:45.528826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:45.529444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:45.529907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:45.532439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:45.596011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:45.661689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:47:45.663569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07061f8-0a11-4ae0-a628-6766f24c1932 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:47:45.686421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 650/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:47:48.152404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0f89-bc3f-4495-9f13-9b7066733289 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:48.152897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0f89-bc3f-4495-9f13-9b7066733289 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:48.153016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a0f89-bc3f-4495-9f13-9b7066733289 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:48.154366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 662/1312] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:47:48.160318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c215247-f582-430d-8b22-94722063633d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:48.160691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c215247-f582-430d-8b22-94722063633d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:48.160860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c215247-f582-430d-8b22-94722063633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:48.241769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9c215247-f582-430d-8b22-94722063633d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:47:48.242972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c215247-f582-430d-8b22-94722063633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:48.336763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 651/1313] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:47:48.617259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a0f89-bc3f-4495-9f13-9b7066733289 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:47:48.618826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c41a0f89-bc3f-4495-9f13-9b7066733289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:48.721971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a0f89-bc3f-4495-9f13-9b7066733289 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:47:48.789894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:48.792411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:47:48.793242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:48.793710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:48.794025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:48.795870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:48.902251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:47:48.903654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b79f9d-b043-46aa-853c-9c7837f15684 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:47:48.922179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 663/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:47:51.353235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca1cfd-8da7-4a2d-839a-17adc1f3cc03 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:51.353811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca1cfd-8da7-4a2d-839a-17adc1f3cc03 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:51.353811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ca1cfd-8da7-4a2d-839a-17adc1f3cc03 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:51.355115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 652/1315] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:47:51.360263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6653c68-a41e-4e2f-a02d-5c3ce30144b4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:51.360684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6653c68-a41e-4e2f-a02d-5c3ce30144b4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:51.360820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6653c68-a41e-4e2f-a02d-5c3ce30144b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:51.426282 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d6653c68-a41e-4e2f-a02d-5c3ce30144b4 None None] MFA Rules not processed for user `597cf93046794f75bf6b7a42a09fb48c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:47:51.427424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6653c68-a41e-4e2f-a02d-5c3ce30144b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:51.477591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:51.522212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 664/1316] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:47:51 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:47:51.801284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca1cfd-8da7-4a2d-839a-17adc1f3cc03 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:47:51.802810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83ca1cfd-8da7-4a2d-839a-17adc1f3cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:51.812498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:51.903269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ca1cfd-8da7-4a2d-839a-17adc1f3cc03 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:47:51.975919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:47:51.976516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:51.976793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:51.977067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:51.978903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:52.039448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:52.086344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:47:52.087740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f51360-95f7-4bf0-924b-7ade47e0fc74 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:47:52.105472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 653/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:47:51 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:47:54.771486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40b14-bd9b-4786-86d0-8497e9a8ec62 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:54.772065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40b14-bd9b-4786-86d0-8497e9a8ec62 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:54.772065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd40b14-bd9b-4786-86d0-8497e9a8ec62 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:54.773439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 665/1318] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:47:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:47:54.778537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99d42-c1ce-4e17-8607-0db6ae1775a9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:54.778984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99d42-c1ce-4e17-8607-0db6ae1775a9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:54.779084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd99d42-c1ce-4e17-8607-0db6ae1775a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:54.844198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ebd99d42-c1ce-4e17-8607-0db6ae1775a9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:47:54.845399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ebd99d42-c1ce-4e17-8607-0db6ae1775a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:54.940665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 654/1319] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:47:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:47:55.219083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd40b14-bd9b-4786-86d0-8497e9a8ec62 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:47:55.220670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4dd40b14-bd9b-4786-86d0-8497e9a8ec62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:55.322558 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd40b14-bd9b-4786-86d0-8497e9a8ec62 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:47:55.395879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:47:55.396484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:55.396767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:55.397042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:55.398923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:55.406271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:55.504982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:47:55.506345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b8bcf8-e1f8-4b71-a4db-5b6c34e0abaa service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:47:55.524758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 666/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:47:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:47:58.302460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4a4bb-ceb2-4f53-bfd9-770f6a799452 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:58.302980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4a4bb-ceb2-4f53-bfd9-770f6a799452 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:58.302980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4a4bb-ceb2-4f53-bfd9-770f6a799452 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:58.304303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 655/1321] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:47:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:47:58.309488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f6d3-6b9c-4923-809e-075c80525d7d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:58.309918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f6d3-6b9c-4923-809e-075c80525d7d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:58.310046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6f6d3-6b9c-4923-809e-075c80525d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:58.383750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f6d6f6d3-6b9c-4923-809e-075c80525d7d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:47:58.384989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f6d6f6d3-6b9c-4923-809e-075c80525d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:58.445889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:58.489365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 667/1322] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:47:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:47:58.773044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4a4bb-ceb2-4f53-bfd9-770f6a799452 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:47:58.774575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83c4a4bb-ceb2-4f53-bfd9-770f6a799452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:58.795093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:58.880482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4a4bb-ceb2-4f53-bfd9-770f6a799452 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:47:58.953166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:47:58.953771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:47:58.954052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:47:58.954919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:47:58.956567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:47:59.028341 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:47:59.073282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:47:59.074730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0fcdee-315e-4ffc-bea7-99fb7a2d67dc service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:47:59.093753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 656/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:47:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:01.306855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dcc9-da9a-498f-b4a9-c710fb677522 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:01.307386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dcc9-da9a-498f-b4a9-c710fb677522 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:01.307482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7dcc9-da9a-498f-b4a9-c710fb677522 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:01.308815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 668/1324] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:01.314477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a107ca-1597-4140-9b84-a1500d0b6b63 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:01.314873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a107ca-1597-4140-9b84-a1500d0b6b63 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:01.315148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a107ca-1597-4140-9b84-a1500d0b6b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:01.380795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b9a107ca-1597-4140-9b84-a1500d0b6b63 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:01.382071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b9a107ca-1597-4140-9b84-a1500d0b6b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:01.485996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 657/1325] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:01.762222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dcc9-da9a-498f-b4a9-c710fb677522 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:01.763796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4f7dcc9-da9a-498f-b4a9-c710fb677522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:01.866718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7dcc9-da9a-498f-b4a9-c710fb677522 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:01.940850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:01.941459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:01.941733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:01.942001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:01.943631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:01.959633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:02.053671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:02.055113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d21d2a-13fc-48b7-9fb2-780706b3a8c7 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:02.073226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 669/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:04.269227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9559a3-6fb1-47fa-96db-8a276ecc5955 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:04.269727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9559a3-6fb1-47fa-96db-8a276ecc5955 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:04.269727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9559a3-6fb1-47fa-96db-8a276ecc5955 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:04.271036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 658/1327] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:04.276694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceb2e0-91b6-430a-b208-093cd5bce20d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:04.277173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceb2e0-91b6-430a-b208-093cd5bce20d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:04.277281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ceb2e0-91b6-430a-b208-093cd5bce20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:04.345755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b6ceb2e0-91b6-430a-b208-093cd5bce20d None None] MFA Rules not processed for user `597cf93046794f75bf6b7a42a09fb48c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:04.346891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b6ceb2e0-91b6-430a-b208-093cd5bce20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:04.414871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:04.447619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 670/1328] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:04 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:04.736844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9559a3-6fb1-47fa-96db-8a276ecc5955 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:04.738365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-da9559a3-6fb1-47fa-96db-8a276ecc5955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:04.770193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:04.849807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9559a3-6fb1-47fa-96db-8a276ecc5955 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:04.927305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:04.927944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:04.928216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:04.928483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:04.930053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:05.009859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:05.042230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:05.043700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83edb9d1-883d-4a9d-87ee-c1fd82e4103b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:05.062878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 659/1329] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:07.419446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dd5a4-fd1f-47aa-96b3-ac8ab6c375f1 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:07.419996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dd5a4-fd1f-47aa-96b3-ac8ab6c375f1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:07.419996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532dd5a4-fd1f-47aa-96b3-ac8ab6c375f1 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:07.421837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 671/1330] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:07.426962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ace0d6-6ba8-4713-96dd-9a39c8248c9f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:07.427375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ace0d6-6ba8-4713-96dd-9a39c8248c9f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:07.427488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ace0d6-6ba8-4713-96dd-9a39c8248c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:07.492960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f0ace0d6-6ba8-4713-96dd-9a39c8248c9f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:07.494163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0ace0d6-6ba8-4713-96dd-9a39c8248c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:07.588348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 660/1331] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:07.864374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dd5a4-fd1f-47aa-96b3-ac8ab6c375f1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:07.865906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-532dd5a4-fd1f-47aa-96b3-ac8ab6c375f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:07.966116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532dd5a4-fd1f-47aa-96b3-ac8ab6c375f1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:08.037159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:08.037776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:08.038053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:08.038321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:08.039954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:08.068143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:08.148729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:08.150109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae19e9-0824-41b8-a4b1-f5089d139ad0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:08.168419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 672/1332] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:10.364849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddf056-a8ab-4b7e-a39d-d2e9e4a7c255 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:10.365355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddf056-a8ab-4b7e-a39d-d2e9e4a7c255 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:10.365355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ddf056-a8ab-4b7e-a39d-d2e9e4a7c255 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:10.366637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 661/1333] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:10.371898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e4c2b-e918-41ac-9247-6e4f5c7643dd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:10.372310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e4c2b-e918-41ac-9247-6e4f5c7643dd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:10.372458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406e4c2b-e918-41ac-9247-6e4f5c7643dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:10.437622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-406e4c2b-e918-41ac-9247-6e4f5c7643dd None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:10.438834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-406e4c2b-e918-41ac-9247-6e4f5c7643dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:10.510862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:10.532715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 673/1334] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:10.829400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4766b-312f-4410-8a34-429c08d99b05 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:10.829691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4766b-312f-4410-8a34-429c08d99b05 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:10.829968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd4766b-312f-4410-8a34-429c08d99b05 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:10.831310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 662/1335] 10.222.0.17 () {54 vars in 965 bytes} [Wed Sep 7 15:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:10.837011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f94875-780b-4e6c-ab7b-ae53bb1280e0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:10.837303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f94875-780b-4e6c-ab7b-ae53bb1280e0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:10.837579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f94875-780b-4e6c-ab7b-ae53bb1280e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:10.912913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-18f94875-780b-4e6c-ab7b-ae53bb1280e0 None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:10.914119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-18f94875-780b-4e6c-ab7b-ae53bb1280e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:10.987915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 674/1336] 10.222.0.17 () {58 vars in 1062 bytes} [Wed Sep 7 15:48:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:10.996907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4766b-312f-4410-8a34-429c08d99b05 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:10.998933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4cd4766b-312f-4410-8a34-429c08d99b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:11.039326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:11.102069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd4766b-312f-4410-8a34-429c08d99b05 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:11.177695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c38258-1795-414e-9277-80838477c497 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:11.178294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38258-1795-414e-9277-80838477c497 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:11.178566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38258-1795-414e-9277-80838477c497 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:11.178831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38258-1795-414e-9277-80838477c497 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:11.180435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-81c38258-1795-414e-9277-80838477c497 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:11.261250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:11.287144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c38258-1795-414e-9277-80838477c497 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:11.288594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c38258-1795-414e-9277-80838477c497 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:11.306597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 663/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:14.272213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dfa0b-3638-48aa-8d8a-8582bb2473ca None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:14.272792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dfa0b-3638-48aa-8d8a-8582bb2473ca None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:14.272792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1dfa0b-3638-48aa-8d8a-8582bb2473ca None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:14.274324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 675/1338] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:14.280194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122c51f-0541-499a-92c0-a1c1f929374f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:14.280620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122c51f-0541-499a-92c0-a1c1f929374f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:14.280738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1122c51f-0541-499a-92c0-a1c1f929374f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:14.347183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1122c51f-0541-499a-92c0-a1c1f929374f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:14.348355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1122c51f-0541-499a-92c0-a1c1f929374f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:14.443950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 664/1339] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:14.727952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dfa0b-3638-48aa-8d8a-8582bb2473ca None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:14.729615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6f1dfa0b-3638-48aa-8d8a-8582bb2473ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:14.804791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:14.831845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1dfa0b-3638-48aa-8d8a-8582bb2473ca None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:14.905546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:14.906150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:14.906421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:14.906687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:14.908338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:15.012399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:15.014285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d59f322-3c64-440b-a7fb-97c3b08c5004 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:15.032326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 676/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:19.174009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da1ef7-6be1-4c85-9b0b-5ef849b0d3e2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:19.174490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da1ef7-6be1-4c85-9b0b-5ef849b0d3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:19.174609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da1ef7-6be1-4c85-9b0b-5ef849b0d3e2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:19.175917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 665/1341] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:19.181280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271a6b7-3629-40b0-b698-301d00645090 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:19.181668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271a6b7-3629-40b0-b698-301d00645090 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:19.181777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d271a6b7-3629-40b0-b698-301d00645090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:19.187902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:19.250565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d271a6b7-3629-40b0-b698-301d00645090 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:19.251870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d271a6b7-3629-40b0-b698-301d00645090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:19.346806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 677/1342] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:19.626876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da1ef7-6be1-4c85-9b0b-5ef849b0d3e2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:19.628448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d0da1ef7-6be1-4c85-9b0b-5ef849b0d3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:19.676871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:19.734254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da1ef7-6be1-4c85-9b0b-5ef849b0d3e2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:19.806539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:19.807150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:19.807430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:19.807729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:19.809318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:19.901341 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:19.918140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:19.919653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3cd14-5fd9-4962-b150-be2f2febadaa service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:19.938063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 666/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:22.684360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ce4a9-192a-4b46-8e53-23575eeda2c2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:22.684955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ce4a9-192a-4b46-8e53-23575eeda2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:22.684955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ce4a9-192a-4b46-8e53-23575eeda2c2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:22.686828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 678/1344] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:22.692021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604051-fc50-4e4b-8e1a-4e271c0bac26 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:22.692484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604051-fc50-4e4b-8e1a-4e271c0bac26 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:22.692595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604051-fc50-4e4b-8e1a-4e271c0bac26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:22.759804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-47604051-fc50-4e4b-8e1a-4e271c0bac26 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:22.761032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-47604051-fc50-4e4b-8e1a-4e271c0bac26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:22.861343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 667/1345] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:23.143223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ce4a9-192a-4b46-8e53-23575eeda2c2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:23.144838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-466ce4a9-192a-4b46-8e53-23575eeda2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:23.237625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:23.257162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ce4a9-192a-4b46-8e53-23575eeda2c2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:23.330696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:23.331306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:23.331609 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:23.331879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:23.333488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:23.441877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:23.443339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd11b7-ec8f-4e52-b333-a4bb367a61d5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:23.462471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 679/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:27.767366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055611e-8dc7-4dcd-90d2-c163de78a135 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:27.767906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055611e-8dc7-4dcd-90d2-c163de78a135 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:27.768029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b055611e-8dc7-4dcd-90d2-c163de78a135 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:27.769329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 668/1347] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:27.774458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c283d-e68c-48d7-a325-d54962247971 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:27.774884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c283d-e68c-48d7-a325-d54962247971 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:27.775041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c283d-e68c-48d7-a325-d54962247971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:27.795055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:27.849831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-499c283d-e68c-48d7-a325-d54962247971 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:27.850872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-499c283d-e68c-48d7-a325-d54962247971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:27.952716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 680/1348] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:48:28.234468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055611e-8dc7-4dcd-90d2-c163de78a135 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:28.236044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b055611e-8dc7-4dcd-90d2-c163de78a135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:28.303579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:28.353016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b055611e-8dc7-4dcd-90d2-c163de78a135 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:28.425289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:28.425892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:28.426171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:28.426439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:28.428062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:28.531597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:28.535690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:28.537126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2e313-98dd-4a51-bc30-3800cc556cd7 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:28.555041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 669/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:33.426399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c4dfe-63e9-42a9-8804-0069f17e22f0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:33.426840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c4dfe-63e9-42a9-8804-0069f17e22f0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:33.426944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c4dfe-63e9-42a9-8804-0069f17e22f0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:33.428900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 681/1350] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:33.433956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee48cc-803e-4a2f-822b-5ad33b7ce8bc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:33.434382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee48cc-803e-4a2f-822b-5ad33b7ce8bc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:33.434530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ee48cc-803e-4a2f-822b-5ad33b7ce8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:33.581543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a8ee48cc-803e-4a2f-822b-5ad33b7ce8bc None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:33.582905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a8ee48cc-803e-4a2f-822b-5ad33b7ce8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:33.721250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 670/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:48:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:37.873640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896bd69-4426-4f9f-a9f8-294ad26abe78 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:37.874169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896bd69-4426-4f9f-a9f8-294ad26abe78 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:37.874169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896bd69-4426-4f9f-a9f8-294ad26abe78 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:37.875683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 682/1352] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:37.880816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaef4be-bf27-417e-875b-9fb77b5ffb81 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:37.881227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaef4be-bf27-417e-875b-9fb77b5ffb81 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:37.881442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaef4be-bf27-417e-875b-9fb77b5ffb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:37.938767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:37.950921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=59 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:37.965326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-afaef4be-bf27-417e-875b-9fb77b5ffb81 None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:37.966558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-afaef4be-bf27-417e-875b-9fb77b5ffb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:38.077048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 671/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:48:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:39.412356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f08d-4042-41ba-a423-d14a3ba9faec None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:39.412894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f08d-4042-41ba-a423-d14a3ba9faec None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:39.412991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99f08d-4042-41ba-a423-d14a3ba9faec None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:39.414354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 683/1354] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:48:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:39.421577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80f71f-05f4-4b07-9b02-d04b852ad54f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:39.422059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80f71f-05f4-4b07-9b02-d04b852ad54f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:39.422157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80f71f-05f4-4b07-9b02-d04b852ad54f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:39.499140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4e80f71f-05f4-4b07-9b02-d04b852ad54f None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:39.500368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4e80f71f-05f4-4b07-9b02-d04b852ad54f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:39.541945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:39.609967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 672/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:48:43.159782 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7ce6-212c-4478-835e-1c58b322d8a8 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:43.160616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7ce6-212c-4478-835e-1c58b322d8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:43.161064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f7ce6-212c-4478-835e-1c58b322d8a8 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:43.162889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 684/1356] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:43.169533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb75b-fae8-4f9b-a416-be29594e7536 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:43.170454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb75b-fae8-4f9b-a416-be29594e7536 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:43.170937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075bb75b-fae8-4f9b-a416-be29594e7536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:43.244966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-075bb75b-fae8-4f9b-a416-be29594e7536 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:43.246175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-075bb75b-fae8-4f9b-a416-be29594e7536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:43.351703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 673/1357] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:43.660969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b19c-daca-4c95-91fa-4ec4566a9f27 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:43.661490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b19c-daca-4c95-91fa-4ec4566a9f27 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:43.661927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f85b19c-daca-4c95-91fa-4ec4566a9f27 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:43.663428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 685/1358] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 15:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:43.684695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f3e8-d94b-41ef-9fc8-8302dac5f999 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:43.685235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f3e8-d94b-41ef-9fc8-8302dac5f999 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:43.685746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e4f3e8-d94b-41ef-9fc8-8302dac5f999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:43.708019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:43.775082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-34e4f3e8-d94b-41ef-9fc8-8302dac5f999 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:43.776314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-34e4f3e8-d94b-41ef-9fc8-8302dac5f999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:43.899616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 674/1359] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 15:48:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:48:43.911725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b19c-daca-4c95-91fa-4ec4566a9f27 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:43.913785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2f85b19c-daca-4c95-91fa-4ec4566a9f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:44.066195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f85b19c-daca-4c95-91fa-4ec4566a9f27 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:44.144470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:44.198627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:44.199963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:44.200938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:44.201359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:44.203279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:44.305630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:44.307427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b603542-034a-47da-8d33-d75b553c98d7 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:44.327164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 686/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:48:46.709318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341c70b-b0ec-4ea6-9f89-40a570a30fc6 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:46.709741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341c70b-b0ec-4ea6-9f89-40a570a30fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:46.710065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341c70b-b0ec-4ea6-9f89-40a570a30fc6 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:46.711319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 675/1361] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:46.717969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a63a0-aa77-49da-81a5-9894d66b8c18 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:46.717969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a63a0-aa77-49da-81a5-9894d66b8c18 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:46.718865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a63a0-aa77-49da-81a5-9894d66b8c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:46.788290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fc5a63a0-aa77-49da-81a5-9894d66b8c18 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:46.789616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fc5a63a0-aa77-49da-81a5-9894d66b8c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:46.831052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:46.895471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 687/1362] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:48:47.182359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341c70b-b0ec-4ea6-9f89-40a570a30fc6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:47.183996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5341c70b-b0ec-4ea6-9f89-40a570a30fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:47.254696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:47.362526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341c70b-b0ec-4ea6-9f89-40a570a30fc6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:47.503323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:47.504026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:47.504303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:47.504574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:47.506162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:47.529187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:47.600504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:47.602208 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af54fd-6c2f-40e7-b374-46af36731786 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:47.620839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 676/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:50.068625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c4aa3-613e-4d68-a526-62ea7f3adb71 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:50.069021 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c4aa3-613e-4d68-a526-62ea7f3adb71 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:50.069797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c4aa3-613e-4d68-a526-62ea7f3adb71 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:50.071755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 688/1364] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:50.077017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437ccb-cd50-463a-98c3-3b59176b49be None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:50.077460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437ccb-cd50-463a-98c3-3b59176b49be None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:50.077584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21437ccb-cd50-463a-98c3-3b59176b49be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:50.145320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-21437ccb-cd50-463a-98c3-3b59176b49be None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:50.146487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-21437ccb-cd50-463a-98c3-3b59176b49be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:50.252624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:50.263709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 677/1365] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:50.545098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f8aa8-6c23-4601-af59-790db3bb8db5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:50.545633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f8aa8-6c23-4601-af59-790db3bb8db5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:50.545973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835f8aa8-6c23-4601-af59-790db3bb8db5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:50.547329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 689/1366] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 15:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:48:50.553188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544d75-fa92-4384-8584-7c9f9ff7de42 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:50.553558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544d75-fa92-4384-8584-7c9f9ff7de42 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:50.553898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21544d75-fa92-4384-8584-7c9f9ff7de42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:50.633326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-21544d75-fa92-4384-8584-7c9f9ff7de42 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:50.634530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-21544d75-fa92-4384-8584-7c9f9ff7de42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:50.756792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 678/1367] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 15:48:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:50.765691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f8aa8-6c23-4601-af59-790db3bb8db5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:50.767179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-835f8aa8-6c23-4601-af59-790db3bb8db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:50.934072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835f8aa8-6c23-4601-af59-790db3bb8db5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:50.955967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:51.054352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:51.055213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:51.055498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:51.055806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:51.057526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:51.154080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:51.155776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d43ea7-2b3b-4f5d-9fae-950959575b70 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:51.178907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 690/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:51.331952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3862d-f432-4322-b785-f13860988b6e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:51.332540 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3862d-f432-4322-b785-f13860988b6e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:51.332841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f3862d-f432-4322-b785-f13860988b6e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:51.334470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 679/1369] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:51.340734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b36ed-05ca-48ed-a4e0-fd2dc6f25b1a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:51.341329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b36ed-05ca-48ed-a4e0-fd2dc6f25b1a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:51.341621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b36ed-05ca-48ed-a4e0-fd2dc6f25b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:51.385534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:51.435943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c52b36ed-05ca-48ed-a4e0-fd2dc6f25b1a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:51.437597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c52b36ed-05ca-48ed-a4e0-fd2dc6f25b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:51.548463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 691/1370] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:51.880296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d05130-51ff-4fea-b80a-4bddcad64642 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:51.880582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d05130-51ff-4fea-b80a-4bddcad64642 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:51.880856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d05130-51ff-4fea-b80a-4bddcad64642 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:51.882400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 680/1371] 10.222.0.17 () {54 vars in 964 bytes} [Wed Sep 7 15:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:51.887129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcba99-dbf3-47e7-9411-c62c6230b40a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:51.888181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcba99-dbf3-47e7-9411-c62c6230b40a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:51.888591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcba99-dbf3-47e7-9411-c62c6230b40a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:51.970905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f5dcba99-dbf3-47e7-9411-c62c6230b40a None None] MFA Rules not processed for user `396fc6e85a65437898bc9d964e429de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:51.972283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f5dcba99-dbf3-47e7-9411-c62c6230b40a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:52.025841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:52.058948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 692/1372] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Sep 7 15:48:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:48:52.067155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d05130-51ff-4fea-b80a-4bddcad64642 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:52.069167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-87d05130-51ff-4fea-b80a-4bddcad64642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:52.123152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:52.190386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d05130-51ff-4fea-b80a-4bddcad64642 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:52.279419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:52.280228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:52.280629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:52.280998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:52.282952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:52.386924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:52.401985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:52.404783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eff480-a64e-40e6-9951-876f3b5c8a7d service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:52.434674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 681/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:53.408111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e6c84-934c-4fbc-9ce4-7fbc6dc9d514 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:53.408650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e6c84-934c-4fbc-9ce4-7fbc6dc9d514 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:53.408859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e6c84-934c-4fbc-9ce4-7fbc6dc9d514 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:53.410356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 693/1374] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:53.415981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4786fc2-5819-48e0-a1f4-fb4f64e10e77 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:53.416402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4786fc2-5819-48e0-a1f4-fb4f64e10e77 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:53.416515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4786fc2-5819-48e0-a1f4-fb4f64e10e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:53.486177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a4786fc2-5819-48e0-a1f4-fb4f64e10e77 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:53.487405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4786fc2-5819-48e0-a1f4-fb4f64e10e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:53.585392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 682/1375] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:53.874097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e6c84-934c-4fbc-9ce4-7fbc6dc9d514 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:53.875677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-129e6c84-934c-4fbc-9ce4-7fbc6dc9d514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:54.036327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e6c84-934c-4fbc-9ce4-7fbc6dc9d514 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:54.092217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:54.158586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:54.159427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:54.159520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:54.159865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:54.161506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:54.254834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:54.256490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7ae8d-32db-4230-ba5c-89e2faed9f0f service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:54.275363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 694/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:56.514975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a37a-e5b1-4ee2-8ca7-eb16f4dab105 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:56.515590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a37a-e5b1-4ee2-8ca7-eb16f4dab105 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:56.515590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050a37a-e5b1-4ee2-8ca7-eb16f4dab105 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:56.517050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 683/1377] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:56.522091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0735bef-567a-4864-9e81-279f78e1c0af None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:56.522477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0735bef-567a-4864-9e81-279f78e1c0af None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:56.522648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0735bef-567a-4864-9e81-279f78e1c0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:56.600937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c0735bef-567a-4864-9e81-279f78e1c0af None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:56.602671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0735bef-567a-4864-9e81-279f78e1c0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:56.611755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:56.719694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 695/1378] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:48:57.014507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a37a-e5b1-4ee2-8ca7-eb16f4dab105 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:48:57.016318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f050a37a-e5b1-4ee2-8ca7-eb16f4dab105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:57.080619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:57.178991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f050a37a-e5b1-4ee2-8ca7-eb16f4dab105 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:48:57.297789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:48:57.298324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:57.298606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:57.298882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:57.300533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:57.320578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:57.396644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:48:57.398296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe0a6cb-b125-4057-b2f2-7199f041ba0f service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:48:57.417627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 684/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:48:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:48:59.620247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f318770-557a-44c3-be4b-3ca692249d65 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:59.620766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f318770-557a-44c3-be4b-3ca692249d65 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:59.620766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f318770-557a-44c3-be4b-3ca692249d65 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:59.622822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 696/1380] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:48:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:48:59.628636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33f445-b2d9-4bb6-9852-3da1012afc13 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:48:59.629052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33f445-b2d9-4bb6-9852-3da1012afc13 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:48:59.629183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33f445-b2d9-4bb6-9852-3da1012afc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:48:59.698480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1d33f445-b2d9-4bb6-9852-3da1012afc13 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:48:59.700287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d33f445-b2d9-4bb6-9852-3da1012afc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:48:59.810045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:48:59.832570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 685/1381] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:48:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:00.114971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f318770-557a-44c3-be4b-3ca692249d65 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:00.116565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f318770-557a-44c3-be4b-3ca692249d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:00.259816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:00.272081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f318770-557a-44c3-be4b-3ca692249d65 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:00.397604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:00.398423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:00.398762 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:00.399097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:00.401206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:00.518736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:00.520340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa372d8-5026-467b-9be5-d6a3a023f758 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:00.533911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:00.544297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 697/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:02.740877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcc527-a334-4164-ae57-c1d44812e7fc None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:02.741407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcc527-a334-4164-ae57-c1d44812e7fc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:02.741567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcc527-a334-4164-ae57-c1d44812e7fc None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:02.743299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 686/1383] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:02.749319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbc29e-8ff2-49c3-9f1e-c64e293993dd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:02.749747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbc29e-8ff2-49c3-9f1e-c64e293993dd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:02.749922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fbc29e-8ff2-49c3-9f1e-c64e293993dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:02.819085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b2fbc29e-8ff2-49c3-9f1e-c64e293993dd None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:02.820829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b2fbc29e-8ff2-49c3-9f1e-c64e293993dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:02.927588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 698/1384] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:03.218431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcc527-a334-4164-ae57-c1d44812e7fc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:03.220593 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-30fcc527-a334-4164-ae57-c1d44812e7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:03.372395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fcc527-a334-4164-ae57-c1d44812e7fc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:03.444295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:03.499500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:03.500472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:03.500952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:03.501303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:03.504068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:03.599854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:03.601473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194b213-69aa-4ad0-9369-7d8ad60f82ca service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:03.620383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 687/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:05.821409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f68dc-e497-43a9-8e21-e50e4f14aaf0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:05.821978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f68dc-e497-43a9-8e21-e50e4f14aaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:05.821978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f68dc-e497-43a9-8e21-e50e4f14aaf0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:05.823329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 699/1386] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:05.829357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0944b-01a3-4b9c-be74-71b7ee754b4f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:05.829851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0944b-01a3-4b9c-be74-71b7ee754b4f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:05.830034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0944b-01a3-4b9c-be74-71b7ee754b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:05.895631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-59d0944b-01a3-4b9c-be74-71b7ee754b4f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:05.896855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-59d0944b-01a3-4b9c-be74-71b7ee754b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:05.931494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:06.007605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 688/1387] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:06.304776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f68dc-e497-43a9-8e21-e50e4f14aaf0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:06.306292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ec8f68dc-e497-43a9-8e21-e50e4f14aaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:06.385580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:06.459084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f68dc-e497-43a9-8e21-e50e4f14aaf0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:06.570283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46315298-a561-46a1-a852-6514e396883e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:06.570887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46315298-a561-46a1-a852-6514e396883e service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:06.571166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46315298-a561-46a1-a852-6514e396883e service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:06.571437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46315298-a561-46a1-a852-6514e396883e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:06.573927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-46315298-a561-46a1-a852-6514e396883e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:06.613479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:06.664882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46315298-a561-46a1-a852-6514e396883e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:06.666498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46315298-a561-46a1-a852-6514e396883e service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:06.684338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 700/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:07.666034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf34537-21bc-4227-835d-634836e4e949 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:07.666575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf34537-21bc-4227-835d-634836e4e949 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:07.666674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf34537-21bc-4227-835d-634836e4e949 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:07.668211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 689/1389] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:07.673464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71299f-8eeb-4685-9ff8-7d4e96eb9b2e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:07.673875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71299f-8eeb-4685-9ff8-7d4e96eb9b2e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:07.674015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d71299f-8eeb-4685-9ff8-7d4e96eb9b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:07.738994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8d71299f-8eeb-4685-9ff8-7d4e96eb9b2e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:07.740224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8d71299f-8eeb-4685-9ff8-7d4e96eb9b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:07.834817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 701/1390] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:08.129510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9dea7-65ff-4217-8340-0a0ab2d86da2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:08.129869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9dea7-65ff-4217-8340-0a0ab2d86da2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:08.130212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9dea7-65ff-4217-8340-0a0ab2d86da2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:08.131659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 690/1391] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 15:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:08.137429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac0dc-8c3b-46f1-b9b6-81ce7d535768 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:08.137717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac0dc-8c3b-46f1-b9b6-81ce7d535768 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:08.137989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac0dc-8c3b-46f1-b9b6-81ce7d535768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:08.154341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:08.216684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2b6ac0dc-8c3b-46f1-b9b6-81ce7d535768 None None] MFA Rules not processed for user `257da2327d944095b51df9f677b3a99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:08.217888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2b6ac0dc-8c3b-46f1-b9b6-81ce7d535768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:08.294058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 702/1392] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:49:08.303027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9dea7-65ff-4217-8340-0a0ab2d86da2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:08.304576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0bd9dea7-65ff-4217-8340-0a0ab2d86da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:08.408898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9dea7-65ff-4217-8340-0a0ab2d86da2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:08.469981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:08.488160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:08.488769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:08.489074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:08.489342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:08.490971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:08.601591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:08.603288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c038c435-0a4a-41bb-866e-41e4cf8dfda0 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:08.622156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 691/1393] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:08.888225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed679a-3f40-4eb6-8945-dd0d7ef9e0c5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:08.888766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed679a-3f40-4eb6-8945-dd0d7ef9e0c5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:08.888766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed679a-3f40-4eb6-8945-dd0d7ef9e0c5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:08.890241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 703/1394] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:08.895926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:08.896374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:08.896482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:08.966711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:08.967989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:09.013030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:09.076593 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 692/1395] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:09.363414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896eb5ca-2b8b-47ec-b209-02130f587a4d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:09.363758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896eb5ca-2b8b-47ec-b209-02130f587a4d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:09.364038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896eb5ca-2b8b-47ec-b209-02130f587a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:09.370648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:09.372539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:09.433234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-896eb5ca-2b8b-47ec-b209-02130f587a4d None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:09.434451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-896eb5ca-2b8b-47ec-b209-02130f587a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:09.488933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47f624-9563-43b4-b5bb-d4c14e90aa3a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:09.495857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:09.535203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 704/1396] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:09.555196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:09.585719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e7f7-967f-4293-864b-d27fa745c256 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:09.586107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e7f7-967f-4293-864b-d27fa745c256 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:09.586448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e7f7-967f-4293-864b-d27fa745c256 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:09.587918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 705/1397] 10.222.0.17 () {54 vars in 943 bytes} [Wed Sep 7 15:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:49:09.595090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd163969-f6cd-4f50-a4a9-8641555b4830 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:09.595479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd163969-f6cd-4f50-a4a9-8641555b4830 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:09.595902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd163969-f6cd-4f50-a4a9-8641555b4830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:09.635611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:09.636149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:09.636426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:09.637195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:09.639052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:09.674780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fd163969-f6cd-4f50-a4a9-8641555b4830 None None] MFA Rules not processed for user `f92a1a83dc5a4ababdebb85429de82b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:09.676160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fd163969-f6cd-4f50-a4a9-8641555b4830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:09.732280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:09.734080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:09.752380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 693/1398] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:49:09.788440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 706/1399] 10.222.0.17 () {58 vars in 1040 bytes} [Wed Sep 7 15:49:09 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:09.800770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:09.802013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:09.875754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:09.905340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c250d-ffdf-44b0-8c51-d6f146adc5c2 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:10.025188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:10.025808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:10.026089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:10.026358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:10.028066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:10.120710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:10.123020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d52083e-6a14-4436-80a2-36c5da5f040c service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:10.132325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 694/1400] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:49:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:49:12.001942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4f7ef-aead-469e-9f2f-44e0551a53e2 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:12.002528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4f7ef-aead-469e-9f2f-44e0551a53e2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:12.002528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4f7ef-aead-469e-9f2f-44e0551a53e2 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:12.004036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 707/1401] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:12.009267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e742092-3ef3-4d9e-8070-a8864a227611 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:12.009702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e742092-3ef3-4d9e-8070-a8864a227611 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:12.009896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e742092-3ef3-4d9e-8070-a8864a227611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:12.025067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:12.086829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2e742092-3ef3-4d9e-8070-a8864a227611 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:12.088243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2e742092-3ef3-4d9e-8070-a8864a227611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:12.221019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 695/1402] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:12.511042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4f7ef-aead-469e-9f2f-44e0551a53e2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:12.512736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9ee4f7ef-aead-469e-9f2f-44e0551a53e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:12.553526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:12.685562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4f7ef-aead-469e-9f2f-44e0551a53e2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:12.800489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:12.811421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:12.812067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:12.812349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:12.812620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:12.814241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:12.914637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:12.916304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8253e8d3-db06-484a-a306-5f701ef6bc90 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:12.937833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 708/1403] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:15.295645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed61d05-2b6b-45f7-b596-d3fb5436f81b None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:15.295645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed61d05-2b6b-45f7-b596-d3fb5436f81b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:15.295645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed61d05-2b6b-45f7-b596-d3fb5436f81b None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:15.295645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 696/1404] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:15.298991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a27db2-e4a2-4589-bc89-2f56161f1b9b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:15.299979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a27db2-e4a2-4589-bc89-2f56161f1b9b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:15.300479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a27db2-e4a2-4589-bc89-2f56161f1b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:15.377941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f5a27db2-e4a2-4589-bc89-2f56161f1b9b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:15.379409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f5a27db2-e4a2-4589-bc89-2f56161f1b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:15.447601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:15.487577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 709/1405] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:15.782117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed61d05-2b6b-45f7-b596-d3fb5436f81b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:15.784026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7ed61d05-2b6b-45f7-b596-d3fb5436f81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:15.878080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:15.941956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed61d05-2b6b-45f7-b596-d3fb5436f81b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:16.067758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:16.068396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:16.072749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:16.072749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:16.072749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:16.131399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:16.172852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:16.172852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb3763b-9fc7-4d1f-af5e-fe44dd9efdf9 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:16.191432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 697/1406] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:18.390246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cb474-1511-4706-a813-99709ffac192 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:18.390753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cb474-1511-4706-a813-99709ffac192 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:18.390753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971cb474-1511-4706-a813-99709ffac192 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:18.392094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 710/1407] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:18.397338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8960e9a-e609-4c63-8e21-0b22609ab616 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:18.397757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8960e9a-e609-4c63-8e21-0b22609ab616 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:18.397897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8960e9a-e609-4c63-8e21-0b22609ab616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:18.474726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e8960e9a-e609-4c63-8e21-0b22609ab616 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:18.476122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8960e9a-e609-4c63-8e21-0b22609ab616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:18.583089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 698/1408] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:18.871255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cb474-1511-4706-a813-99709ffac192 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:18.872809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-971cb474-1511-4706-a813-99709ffac192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:19.043411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971cb474-1511-4706-a813-99709ffac192 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:19.099513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:19.174835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:19.175466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:19.175810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:19.176089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:19.177859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:19.393095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:19.394714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa84fe-1e3b-44d0-bb04-7bcd0a1ee6e4 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:19.413410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 711/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:28.581058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:29.410843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=76 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:32.963046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095954-036c-459d-957d-fc04f836bf13 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:32.963657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095954-036c-459d-957d-fc04f836bf13 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:32.963657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec095954-036c-459d-957d-fc04f836bf13 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:32.964979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 699/1410] 10.222.0.17 () {54 vars in 918 bytes} [Wed Sep 7 15:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:32.969929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88694fd-aa14-4830-a358-6433ccda99e1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:32.970342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88694fd-aa14-4830-a358-6433ccda99e1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:32.970519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88694fd-aa14-4830-a358-6433ccda99e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:33.044991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f88694fd-aa14-4830-a358-6433ccda99e1 None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:33.046183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f88694fd-aa14-4830-a358-6433ccda99e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:33.149084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 712/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Sep 7 15:49:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:33.163114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386b66c-d11e-4c01-9cc4-0d3614e32ea2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:33.163400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386b66c-d11e-4c01-9cc4-0d3614e32ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:33.163712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386b66c-d11e-4c01-9cc4-0d3614e32ea2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:33.165030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 700/1412] 10.222.0.17 () {54 vars in 918 bytes} [Wed Sep 7 15:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:33.170454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679b55-5254-45d3-af27-5c454e401e7c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:33.170743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679b55-5254-45d3-af27-5c454e401e7c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:33.171019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba679b55-5254-45d3-af27-5c454e401e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:33.242151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:33.244795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ba679b55-5254-45d3-af27-5c454e401e7c None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:33.247110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ba679b55-5254-45d3-af27-5c454e401e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:33.362566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 713/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Sep 7 15:49:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:35.934726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0314d-9e8a-4cfa-b4a0-fc95c8885b27 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:35.935275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0314d-9e8a-4cfa-b4a0-fc95c8885b27 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:35.935275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0314d-9e8a-4cfa-b4a0-fc95c8885b27 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:35.936673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:35.941876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259941-4217-480e-8716-84748ae90fe6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:35.942302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259941-4217-480e-8716-84748ae90fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:35.942408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2259941-4217-480e-8716-84748ae90fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:36.006732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e2259941-4217-480e-8716-84748ae90fe6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:36.007950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e2259941-4217-480e-8716-84748ae90fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:36.060867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:36.101620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:49:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:36.390666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0314d-9e8a-4cfa-b4a0-fc95c8885b27 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:36.392362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fab0314d-9e8a-4cfa-b4a0-fc95c8885b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:36.533981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0314d-9e8a-4cfa-b4a0-fc95c8885b27 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:36.606427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:36.646510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:36.647034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:36.647308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:36.647594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:36.649184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:36.742553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:36.744190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74fe548-d88c-42bb-b1dd-7bf692140ea8 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:36.762383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 702/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:37.905726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325e8c0-345f-4533-b19e-98fe77206a46 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:37.906297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325e8c0-345f-4533-b19e-98fe77206a46 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:37.906297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325e8c0-345f-4533-b19e-98fe77206a46 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:37.907706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:37.913696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9af85-825e-486f-a145-2beae55be738 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:37.914121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9af85-825e-486f-a145-2beae55be738 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:37.914230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb9af85-825e-486f-a145-2beae55be738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:37.989470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fdb9af85-825e-486f-a145-2beae55be738 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:37.990686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fdb9af85-825e-486f-a145-2beae55be738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:38.016693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:38.084731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:38.234487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325e8c0-345f-4533-b19e-98fe77206a46 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:38.236032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9325e8c0-345f-4533-b19e-98fe77206a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:38.383906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325e8c0-345f-4533-b19e-98fe77206a46 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:38.423458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:38.495933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:38.496540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:38.496818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:38.497084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:38.498677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:38.590936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:38.592572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca39cec3-4d23-4226-9f8a-069ed6a840e1 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:38.610654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 716/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:39.735934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6fbe6-0e45-4642-b179-379cea02b1f5 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:39.736532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6fbe6-0e45-4642-b179-379cea02b1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:39.736532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d6fbe6-0e45-4642-b179-379cea02b1f5 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:39.737825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:39.744031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99fcc9-7399-4f6a-b758-d965d1814620 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:39.744475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99fcc9-7399-4f6a-b758-d965d1814620 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:39.744623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b99fcc9-7399-4f6a-b758-d965d1814620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:39.808992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:39.832980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8b99fcc9-7399-4f6a-b758-d965d1814620 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:39.834198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8b99fcc9-7399-4f6a-b758-d965d1814620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:39.929210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:40.078821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6fbe6-0e45-4642-b179-379cea02b1f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:40.080305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5d6fbe6-0e45-4642-b179-379cea02b1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:40.224355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d6fbe6-0e45-4642-b179-379cea02b1f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:40.236970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:40.335907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:40.336432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:40.336706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:40.336972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:40.338542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:40.426543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:40.428175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e074f669-624a-4ab9-aeb8-0ee3731afa57 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:40.445682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 705/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:41.547349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb1884-b93e-4d84-80cc-b3360c8a4d50 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:41.547882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb1884-b93e-4d84-80cc-b3360c8a4d50 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:41.547882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb1884-b93e-4d84-80cc-b3360c8a4d50 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:41.549197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:41.555535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364611c-e726-499b-8675-feeb2f4ce987 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:41.555994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364611c-e726-499b-8675-feeb2f4ce987 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:41.556104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6364611c-e726-499b-8675-feeb2f4ce987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:41.575458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:41.633787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6364611c-e726-499b-8675-feeb2f4ce987 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:41.635002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6364611c-e726-499b-8675-feeb2f4ce987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:41.730742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:41.881519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb1884-b93e-4d84-80cc-b3360c8a4d50 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:41.882967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b9cb1884-b93e-4d84-80cc-b3360c8a4d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:41.989525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:42.024659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb1884-b93e-4d84-80cc-b3360c8a4d50 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:42.136489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:42.137035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:42.137309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:42.137578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:42.139155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:42.212723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:42.231054 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:42.232691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9e864-ca66-4133-9a2c-7489c5d2e85d service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:42.251100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 719/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:43.364523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cff3a-47f0-45ad-8f2a-9a0101fa60d2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:43.365076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cff3a-47f0-45ad-8f2a-9a0101fa60d2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:43.365076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cff3a-47f0-45ad-8f2a-9a0101fa60d2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:43.366484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:43.372928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef1dc5-dbbc-45e3-b1ae-e844d6f7fc36 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:43.373391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef1dc5-dbbc-45e3-b1ae-e844d6f7fc36 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:43.373580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ef1dc5-dbbc-45e3-b1ae-e844d6f7fc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:43.447286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f6ef1dc5-dbbc-45e3-b1ae-e844d6f7fc36 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:43.448959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f6ef1dc5-dbbc-45e3-b1ae-e844d6f7fc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:43.544055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:43.693271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cff3a-47f0-45ad-8f2a-9a0101fa60d2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:43.694856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-275cff3a-47f0-45ad-8f2a-9a0101fa60d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:43.779443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:43.845562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cff3a-47f0-45ad-8f2a-9a0101fa60d2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:43.958380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:43.958991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:43.959267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:43.959534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:43.961167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:44.009581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:44.050676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:44.052204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e82b928-5f87-4adc-a890-becd868fbd69 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:44.070221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 708/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:45.110659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670b8c-f97b-4bfe-9056-46b1bb74e684 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:45.111184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670b8c-f97b-4bfe-9056-46b1bb74e684 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:45.111184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b670b8c-f97b-4bfe-9056-46b1bb74e684 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:45.112519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:45.118526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad7a2b-f251-4218-a680-09e8af188574 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:45.118909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad7a2b-f251-4218-a680-09e8af188574 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:45.119085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ad7a2b-f251-4218-a680-09e8af188574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:45.193893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-11ad7a2b-f251-4218-a680-09e8af188574 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:45.195116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-11ad7a2b-f251-4218-a680-09e8af188574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:45.293998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:45.442891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670b8c-f97b-4bfe-9056-46b1bb74e684 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:45.444482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7b670b8c-f97b-4bfe-9056-46b1bb74e684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:45.492200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:45.593532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b670b8c-f97b-4bfe-9056-46b1bb74e684 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:45.704688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:45.705217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:45.705494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:45.705763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:45.707333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:45.714135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:45.796200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:45.797805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cc8740-b361-4d37-9580-b42b3e2a9e46 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:45.816453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 722/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:46.873506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeca8e-161f-42dc-abad-98a97a3fdce8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:46.874024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeca8e-161f-42dc-abad-98a97a3fdce8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:46.874024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdeca8e-161f-42dc-abad-98a97a3fdce8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:46.875318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:46.881764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636185c-4d0d-449e-b1c8-c2c50c14a1c6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:46.882214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636185c-4d0d-449e-b1c8-c2c50c14a1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:46.882324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0636185c-4d0d-449e-b1c8-c2c50c14a1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:46.956639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0636185c-4d0d-449e-b1c8-c2c50c14a1c6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:46.957854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0636185c-4d0d-449e-b1c8-c2c50c14a1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:47.024329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:47.054125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:47.203475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeca8e-161f-42dc-abad-98a97a3fdce8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:47.205101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5bdeca8e-161f-42dc-abad-98a97a3fdce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:47.223271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:47.348692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdeca8e-161f-42dc-abad-98a97a3fdce8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:47.434672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:47.460726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:47.461323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:47.461597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:47.461864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:47.463429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:47.551828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:47.553421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d17e6-e980-4c16-94ed-4e2aed1359e1 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:47.571353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 711/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:48.624022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848227-5d60-4fbd-9ac6-0ad253c6759b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:48.624581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848227-5d60-4fbd-9ac6-0ad253c6759b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:48.624581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3848227-5d60-4fbd-9ac6-0ad253c6759b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:48.625912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:48.632050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d874-f0bf-4cc3-b1f6-02d5aab7801d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:48.632501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d874-f0bf-4cc3-b1f6-02d5aab7801d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:48.632609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d874-f0bf-4cc3-b1f6-02d5aab7801d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:48.707188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8855d874-f0bf-4cc3-b1f6-02d5aab7801d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:48.708486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8855d874-f0bf-4cc3-b1f6-02d5aab7801d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:48.750178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:48.803203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:48.952035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3848227-5d60-4fbd-9ac6-0ad253c6759b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:48.953567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e3848227-5d60-4fbd-9ac6-0ad253c6759b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:49.096482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3848227-5d60-4fbd-9ac6-0ad253c6759b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:49.150571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:49.210464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:49.211010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:49.211289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:49.211581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:49.213164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:49.302474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:49.304108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a6e2ae-3a0f-4080-88b9-02de7f045a8d service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:49.321867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 725/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:50.359036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c69ea-45fe-4d5b-a873-e2de7bbe71d4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:50.359535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c69ea-45fe-4d5b-a873-e2de7bbe71d4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:50.359535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c69ea-45fe-4d5b-a873-e2de7bbe71d4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:50.360883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:50.367097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6048985-796c-48d9-913d-9820eb8355e8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:50.367559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6048985-796c-48d9-913d-9820eb8355e8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:50.367660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6048985-796c-48d9-913d-9820eb8355e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:50.441526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c6048985-796c-48d9-913d-9820eb8355e8 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:50.442729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c6048985-796c-48d9-913d-9820eb8355e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:50.444764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:50.538671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:50.688280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c69ea-45fe-4d5b-a873-e2de7bbe71d4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:50.689840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f96c69ea-45fe-4d5b-a873-e2de7bbe71d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:50.833354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c69ea-45fe-4d5b-a873-e2de7bbe71d4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:50.860075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:50.944984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:50.945512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:50.945784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:50.946054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:50.947650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:51.034071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:51.035659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa8a9f2-2b53-4276-aef8-56514a42d3eb service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:51.053367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 714/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:53.100319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a787797-be7e-404b-8321-dc4f4a2b773b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:53.101014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a787797-be7e-404b-8321-dc4f4a2b773b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:53.101014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a787797-be7e-404b-8321-dc4f4a2b773b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:53.102254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:49:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:53.108405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad2871-51ec-47b3-9d62-9427f5263f98 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:53.108827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad2871-51ec-47b3-9d62-9427f5263f98 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:53.108934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ad2871-51ec-47b3-9d62-9427f5263f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:53.149801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:53.176562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-57ad2871-51ec-47b3-9d62-9427f5263f98 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:53.177811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-57ad2871-51ec-47b3-9d62-9427f5263f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:53.273634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:49:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:53.553737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a787797-be7e-404b-8321-dc4f4a2b773b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:53.555310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7a787797-be7e-404b-8321-dc4f4a2b773b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:53.661627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a787797-be7e-404b-8321-dc4f4a2b773b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:53.693538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:53.735195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:53.735836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:53.736121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:53.736396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:53.737999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:53.848159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:53.849536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334900a3-6036-44d0-bdaa-fecc6ee7467a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:53.867917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 728/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:56.162515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90192-bd61-4ecd-b17c-b9371244ff9c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:56.162940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90192-bd61-4ecd-b17c-b9371244ff9c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:56.163063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f90192-bd61-4ecd-b17c-b9371244ff9c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:56.164421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:49:56.169701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9d176-94bf-4300-bd24-c9507b48b0f9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:56.170103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9d176-94bf-4300-bd24-c9507b48b0f9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:56.170266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9d176-94bf-4300-bd24-c9507b48b0f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:56.236056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-04b9d176-94bf-4300-bd24-c9507b48b0f9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:56.237257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-04b9d176-94bf-4300-bd24-c9507b48b0f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:56.249871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:56.341590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:49:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:56.624387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90192-bd61-4ecd-b17c-b9371244ff9c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:56.625903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-12f90192-bd61-4ecd-b17c-b9371244ff9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:56.726224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f90192-bd61-4ecd-b17c-b9371244ff9c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:49:56.738769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:56.799104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:49:56.799728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:56.800006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:56.800273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:56.801866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:56.908792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:49:56.910223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f690e1e-a09e-40a8-948b-4641cbc25f3f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:49:56.927804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 717/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:49:59.419143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c123-b225-4217-89d2-17a3d3664966 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:59.419716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c123-b225-4217-89d2-17a3d3664966 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:59.419716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c123-b225-4217-89d2-17a3d3664966 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:59.420989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:49:59.426222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bf72f-7f66-4d10-b20c-26e0da30d058 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:49:59.426643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bf72f-7f66-4d10-b20c-26e0da30d058 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:49:59.426839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bf72f-7f66-4d10-b20c-26e0da30d058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:49:59.475403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:49:59.493811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-444bf72f-7f66-4d10-b20c-26e0da30d058 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:49:59.495512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-444bf72f-7f66-4d10-b20c-26e0da30d058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:59.591132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:49:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:49:59.877071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42c123-b225-4217-89d2-17a3d3664966 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:49:59.878602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4b42c123-b225-4217-89d2-17a3d3664966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:49:59.980998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42c123-b225-4217-89d2-17a3d3664966 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:00.020617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:00.053950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:00.054552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:00.054830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:00.055098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:00.056705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:00.165910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:00.167894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9272c54f-bb15-4d32-92c5-97114d43de51 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:00.186973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 731/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:49:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:02.426134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ebfe1-8091-4825-a4ce-1f6a32561a9c None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:02.426682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ebfe1-8091-4825-a4ce-1f6a32561a9c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:02.426682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ebfe1-8091-4825-a4ce-1f6a32561a9c None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:02.428076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:02.433575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568bf53-6c3c-4521-b790-a1c6ce0abc7b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:02.433983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568bf53-6c3c-4521-b790-a1c6ce0abc7b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:02.434108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f568bf53-6c3c-4521-b790-a1c6ce0abc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:02.499672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f568bf53-6c3c-4521-b790-a1c6ce0abc7b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:02.500862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f568bf53-6c3c-4521-b790-a1c6ce0abc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:02.521159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:02.597984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:02.883769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ebfe1-8091-4825-a4ce-1f6a32561a9c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:02.885340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0d4ebfe1-8091-4825-a4ce-1f6a32561a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:02.987315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ebfe1-8091-4825-a4ce-1f6a32561a9c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:03.007164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:03.060957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:03.061552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:03.061824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:03.062089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:03.063686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:03.169609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:03.171047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887e024-16eb-4c80-b94e-f1eda9201f66 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:03.189601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 720/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:05.378399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b670ba-6651-40d7-adae-fc17e550efe5 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:05.378911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b670ba-6651-40d7-adae-fc17e550efe5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:05.378911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b670ba-6651-40d7-adae-fc17e550efe5 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:05.382408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:05.388289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae82f-e279-4b8b-9bfe-edaccf0709ae None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:05.388771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae82f-e279-4b8b-9bfe-edaccf0709ae None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:05.388870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ae82f-e279-4b8b-9bfe-edaccf0709ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:05.452421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:05.455293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-910ae82f-e279-4b8b-9bfe-edaccf0709ae None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:05.457121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-910ae82f-e279-4b8b-9bfe-edaccf0709ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:05.553706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:05.833711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b670ba-6651-40d7-adae-fc17e550efe5 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:05.835246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c4b670ba-6651-40d7-adae-fc17e550efe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:05.938778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b670ba-6651-40d7-adae-fc17e550efe5 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:05.986875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:06.011155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:06.011797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:06.012075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:06.012344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:06.013926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:06.118840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:06.120411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2ed122-4136-4d34-92bf-a8614eab1b07 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:06.140199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 734/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:08.316429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6bb0f-b2d6-4647-a290-5c95e61a0a08 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:08.316983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6bb0f-b2d6-4647-a290-5c95e61a0a08 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:08.316983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6bb0f-b2d6-4647-a290-5c95e61a0a08 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:08.318389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:08.324287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1cdc8-2884-4bb2-bd7e-bcf7019f1ccd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:08.324717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1cdc8-2884-4bb2-bd7e-bcf7019f1ccd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:08.324824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c1cdc8-2884-4bb2-bd7e-bcf7019f1ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:08.389816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-64c1cdc8-2884-4bb2-bd7e-bcf7019f1ccd None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:08.390948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-64c1cdc8-2884-4bb2-bd7e-bcf7019f1ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:08.420422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:08.487819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:08.770576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6bb0f-b2d6-4647-a290-5c95e61a0a08 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:08.772169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-98b6bb0f-b2d6-4647-a290-5c95e61a0a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:08.873898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6bb0f-b2d6-4647-a290-5c95e61a0a08 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:08.900736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:08.946327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:08.946926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:08.947202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:08.947473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:08.949088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:09.055342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:09.056726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf555-2c47-491f-b265-446a77f70ce8 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:09.074730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 723/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:11.283212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bae9f-68cc-4453-97cb-f2a9e5f2c8f6 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:11.283816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bae9f-68cc-4453-97cb-f2a9e5f2c8f6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:11.283938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654bae9f-68cc-4453-97cb-f2a9e5f2c8f6 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:11.285607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:11.292117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24c4e2-3f3d-487e-9598-9e613317ef6e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:11.292583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24c4e2-3f3d-487e-9598-9e613317ef6e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:11.292680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24c4e2-3f3d-487e-9598-9e613317ef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:11.358619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4d24c4e2-3f3d-487e-9598-9e613317ef6e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:11.359868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4d24c4e2-3f3d-487e-9598-9e613317ef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:11.368118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:11.456017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:11.742969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bae9f-68cc-4453-97cb-f2a9e5f2c8f6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:11.744567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-654bae9f-68cc-4453-97cb-f2a9e5f2c8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:11.844960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654bae9f-68cc-4453-97cb-f2a9e5f2c8f6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:11.898839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:11.917515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:11.918104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:11.918383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:11.918651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:11.920279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:12.027078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:12.028514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830a7e4-4e9b-4e00-ad65-e70d1c7d1d07 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:12.046437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 737/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:14.233648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc121-bc22-428d-956d-b495feef411a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:14.234176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc121-bc22-428d-956d-b495feef411a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:14.234176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc121-bc22-428d-956d-b495feef411a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:14.235534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:14.240898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6d33-3447-437c-8e0d-a91632e36467 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:14.241331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6d33-3447-437c-8e0d-a91632e36467 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:14.241485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc6d33-3447-437c-8e0d-a91632e36467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:14.316828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-febc6d33-3447-437c-8e0d-a91632e36467 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:14.318051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-febc6d33-3447-437c-8e0d-a91632e36467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:14.359607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:14.418798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:14.721674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accc121-bc22-428d-956d-b495feef411a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:14.723232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5accc121-bc22-428d-956d-b495feef411a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:14.827750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accc121-bc22-428d-956d-b495feef411a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:14.864742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:14.899441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:14.900063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:14.900356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:14.901142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:14.902712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:15.011008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:15.013074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb8e566-ea7e-4bc5-9dd3-dc8a0fced966 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:15.030908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 726/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:17.193958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b49b58-8adf-4440-b546-22891f0bee6d None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:17.194494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b49b58-8adf-4440-b546-22891f0bee6d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:17.194494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b49b58-8adf-4440-b546-22891f0bee6d None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:17.195916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:17.201225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b0fc-db0b-4928-b0a9-9bda03eae84e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:17.201633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b0fc-db0b-4928-b0a9-9bda03eae84e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:17.201765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61b0fc-db0b-4928-b0a9-9bda03eae84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:17.267534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bd61b0fc-db0b-4928-b0a9-9bda03eae84e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:17.268782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bd61b0fc-db0b-4928-b0a9-9bda03eae84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:17.286329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:17.368485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:17.655680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b49b58-8adf-4440-b546-22891f0bee6d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:17.657225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-90b49b58-8adf-4440-b546-22891f0bee6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:17.759256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b49b58-8adf-4440-b546-22891f0bee6d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:17.824150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:17.830983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:17.831599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:17.831885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:17.832159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:17.833755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:17.940759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:17.942203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c20bca7-1e6c-4f50-8a76-d845df8efa2a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:17.965873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 740/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:20.113110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa634c-43ef-42a4-b11c-8c70024afaa0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:20.113710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa634c-43ef-42a4-b11c-8c70024afaa0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:20.113710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aa634c-43ef-42a4-b11c-8c70024afaa0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:20.114977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:20.120772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a3a6c-d1a9-43fe-b05c-ef09d460c980 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:20.121154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a3a6c-d1a9-43fe-b05c-ef09d460c980 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:20.121323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a3a6c-d1a9-43fe-b05c-ef09d460c980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:20.188812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-159a3a6c-d1a9-43fe-b05c-ef09d460c980 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:20.190024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-159a3a6c-d1a9-43fe-b05c-ef09d460c980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:20.237369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:20.285635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:50:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:20.569678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa634c-43ef-42a4-b11c-8c70024afaa0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:20.571210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7aa634c-43ef-42a4-b11c-8c70024afaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:20.672459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aa634c-43ef-42a4-b11c-8c70024afaa0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:20.720297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:20.746023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:20.746628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:20.746906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:20.747178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:20.748795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:20.855622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:20.856995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d715e93-de11-43d4-8a76-6660962cf38c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:20.874640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 729/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:30.283881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=22 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:30.872385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=66 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:40.689951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb56630-04d5-4cd3-aaf0-17907ff4c5d0 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:40.690514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb56630-04d5-4cd3-aaf0-17907ff4c5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:40.690514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb56630-04d5-4cd3-aaf0-17907ff4c5d0 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:40.691874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 742/1471] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:40.697646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9ed27-7aa8-4443-ba68-689f55f3d974 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:40.698073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9ed27-7aa8-4443-ba68-689f55f3d974 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:40.698221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9ed27-7aa8-4443-ba68-689f55f3d974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:40.763410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-49b9ed27-7aa8-4443-ba68-689f55f3d974 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:40.764634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-49b9ed27-7aa8-4443-ba68-689f55f3d974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:40.861520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 730/1472] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:41.157931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6fd5-5f2b-4554-9c62-377c2ce01d77 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:41.158213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6fd5-5f2b-4554-9c62-377c2ce01d77 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:41.158515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c6fd5-5f2b-4554-9c62-377c2ce01d77 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:41.159884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 743/1473] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 15:50:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:41.164825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb11bf-bc69-434a-99cb-0893e46364ce None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:41.165106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb11bf-bc69-434a-99cb-0893e46364ce None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:41.165955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb11bf-bc69-434a-99cb-0893e46364ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:41.239903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-78eb11bf-bc69-434a-99cb-0893e46364ce None None] MFA Rules not processed for user `257da2327d944095b51df9f677b3a99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:41.241099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-78eb11bf-bc69-434a-99cb-0893e46364ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:41.261024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:41.319719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 731/1474] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 15:50:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:41.327476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6fd5-5f2b-4554-9c62-377c2ce01d77 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:41.329014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a31c6fd5-5f2b-4554-9c62-377c2ce01d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:41.438408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c6fd5-5f2b-4554-9c62-377c2ce01d77 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:41.510854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:41.511616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:41.511898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:41.512195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:41.513810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:41.567006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:41.623688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:41.625140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ddb345-11a8-4a52-8902-4afd00b9b00f service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:41.644079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 744/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:43.860216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b17c-bece-45be-8c05-228f6d446ae4 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:43.860855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b17c-bece-45be-8c05-228f6d446ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:43.860855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7b17c-bece-45be-8c05-228f6d446ae4 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:43.862133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 732/1476] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:43.867255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c2c93-23b6-407c-a75d-d62f4c0f0299 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:43.867714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c2c93-23b6-407c-a75d-d62f4c0f0299 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:43.867829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c2c93-23b6-407c-a75d-d62f4c0f0299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:43.932706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-527c2c93-23b6-407c-a75d-d62f4c0f0299 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:43.933961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-527c2c93-23b6-407c-a75d-d62f4c0f0299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:44.030308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 745/1477] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:44.306155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b17c-bece-45be-8c05-228f6d446ae4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:44.307744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bdc7b17c-bece-45be-8c05-228f6d446ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:44.549458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7b17c-bece-45be-8c05-228f6d446ae4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:44.596886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:44.640930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:44.641536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:44.641814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:44.642082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:44.642719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:44.644487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:50:44.644487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:50:44.644487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:50:44.645609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112806da-04b2-4ad6-a8f4-c2d24bcedb47 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:44.664716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 733/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Sep 7 15:50:44 2022] GET /identity/v3/projects => generated 2553 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:50:46.806826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901d128-3489-41b3-9e8d-d4ea3ef5feec None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:46.807343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901d128-3489-41b3-9e8d-d4ea3ef5feec None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:46.807343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a901d128-3489-41b3-9e8d-d4ea3ef5feec None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:46.808748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 746/1479] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:46.813884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c22e7-9cdc-4c2f-8334-87ba8b320916 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:46.814320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c22e7-9cdc-4c2f-8334-87ba8b320916 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:46.814432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1c22e7-9cdc-4c2f-8334-87ba8b320916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:46.880037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ef1c22e7-9cdc-4c2f-8334-87ba8b320916 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:46.881256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef1c22e7-9cdc-4c2f-8334-87ba8b320916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:46.976056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 734/1480] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:47.268166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901d128-3489-41b3-9e8d-d4ea3ef5feec None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:47.269684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a901d128-3489-41b3-9e8d-d4ea3ef5feec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:47.279612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:47.413331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a901d128-3489-41b3-9e8d-d4ea3ef5feec None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:47.491832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:47.528324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:47.528947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:47.529228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:47.529501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:47.531171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:47.620497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:47.622220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4111373a-9239-4f78-89a6-e31538accc5f service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:47.639963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 747/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:49.944831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb12bb5-11c2-44fc-ba77-4f0db8397bc8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:49.945401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb12bb5-11c2-44fc-ba77-4f0db8397bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:49.945401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb12bb5-11c2-44fc-ba77-4f0db8397bc8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:49.947248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 735/1482] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:49.952557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875501e-6950-4b02-9ae4-f896a85245f8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:49.953000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875501e-6950-4b02-9ae4-f896a85245f8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:49.953107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875501e-6950-4b02-9ae4-f896a85245f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:50.018550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2875501e-6950-4b02-9ae4-f896a85245f8 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:50.019792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2875501e-6950-4b02-9ae4-f896a85245f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:50.065474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:50.116915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 748/1483] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:50.400608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb12bb5-11c2-44fc-ba77-4f0db8397bc8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:50.402155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5bb12bb5-11c2-44fc-ba77-4f0db8397bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:50.425355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:50.505792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb12bb5-11c2-44fc-ba77-4f0db8397bc8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:50.577641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:50.578241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:50.578520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:50.578787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:50.580423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:50.661974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:50.697267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:50.698697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750902a0-5453-4926-9dd4-4e48df5e7fbb service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:50.716362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 736/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:52.908626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a830-700a-49e0-8624-65aa9e14e3f7 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:52.909167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a830-700a-49e0-8624-65aa9e14e3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:52.909167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c212a830-700a-49e0-8624-65aa9e14e3f7 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:52.910528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 749/1485] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:52.916284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809d502-9171-423b-9924-aebc9931e30b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:52.916680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809d502-9171-423b-9924-aebc9931e30b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:52.916833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809d502-9171-423b-9924-aebc9931e30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:52.982074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0809d502-9171-423b-9924-aebc9931e30b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:52.983287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0809d502-9171-423b-9924-aebc9931e30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:53.077213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 737/1486] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:53.363663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212a830-700a-49e0-8624-65aa9e14e3f7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:53.365206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c212a830-700a-49e0-8624-65aa9e14e3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:53.466376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c212a830-700a-49e0-8624-65aa9e14e3f7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:53.536858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:53.538751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:53.539407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:53.539940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:53.540426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:53.542236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:53.653131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:53.654502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f6101-e163-47c7-a3d6-0a9790a76649 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:53.673178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 750/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:55.930187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49555e-d28b-436a-88f0-c066625ed0a2 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:55.931382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49555e-d28b-436a-88f0-c066625ed0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:55.931382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49555e-d28b-436a-88f0-c066625ed0a2 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:55.932700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 738/1488] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:55.938090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d44c6-c109-4555-a1bb-0ae0b7a6fb27 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:55.938519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d44c6-c109-4555-a1bb-0ae0b7a6fb27 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:55.938628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d44c6-c109-4555-a1bb-0ae0b7a6fb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:56.004683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-394d44c6-c109-4555-a1bb-0ae0b7a6fb27 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:56.005882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-394d44c6-c109-4555-a1bb-0ae0b7a6fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:56.057383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:56.099971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 751/1489] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:56.395572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49555e-d28b-436a-88f0-c066625ed0a2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:56.397125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9c49555e-d28b-436a-88f0-c066625ed0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:56.434350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:56.501460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49555e-d28b-436a-88f0-c066625ed0a2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:56.573234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:56.573829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:56.574103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:56.574369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:56.575964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:56.655075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:56.683483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:56.684940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac6056-d5cf-472d-b3cd-2a7a6e93d0ec service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:56.702983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 739/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:50:59.041789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6dc20-7908-49cb-8bbc-86d27f6f2d8c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:59.042359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6dc20-7908-49cb-8bbc-86d27f6f2d8c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:59.042359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6dc20-7908-49cb-8bbc-86d27f6f2d8c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:59.043715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 752/1491] 10.222.0.17 () {54 vars in 927 bytes} [Wed Sep 7 15:50:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:50:59.048841 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6cabd-2ffc-47df-a313-b9f13342b769 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:59.049249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6cabd-2ffc-47df-a313-b9f13342b769 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:59.049397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6cabd-2ffc-47df-a313-b9f13342b769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:59.114286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e2b6cabd-2ffc-47df-a313-b9f13342b769 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:50:59.115516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e2b6cabd-2ffc-47df-a313-b9f13342b769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:59.210524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 740/1492] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Sep 7 15:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:50:59.505582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6dc20-7908-49cb-8bbc-86d27f6f2d8c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:50:59.507156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d9f6dc20-7908-49cb-8bbc-86d27f6f2d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:59.611384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6dc20-7908-49cb-8bbc-86d27f6f2d8c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:50:59.684437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:50:59.685053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:50:59.685335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:50:59.685606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:50:59.687205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:50:59.696858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:50:59.795049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:50:59.796450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3550b3-49fc-48c9-9d4d-445b40c5316b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:50:59.813777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 753/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:51:09.208737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=82 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:51:09.811524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=50 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:18.695467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:18.696067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:18.696067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:18.769808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:52:18.770998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:18.866001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 741/1494] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:52:18.873712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:18.874192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:18.874301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:18.947566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:52:18.948706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:19.041766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 754/1495] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:52:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:52:19.046339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:19.047523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:19.059249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cc4ef2-85b2-4c5e-9b93-38e99b223262 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:19.119185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:19.152309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:19.152912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:19.153187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:19.153457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:19.154100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:52:19.154403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:19.155465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:19.155465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:52:19.155465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:52:19.156370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:19.168201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 742/1496] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:52:19.176325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:19.185956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bbb370-b89b-4d85-9459-3f356e50ad21 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:19.246797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:19.283229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:19.283869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:19.284150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:19.284421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:19.285047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:19.286765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:19.286765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:52:19.286765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:52:19.288511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:19.342289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 755/1497] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:52:19 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:52:19.346423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:19.347994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:19.381773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c7720c-9784-4d11-bccc-195101652bbd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:19.479776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:19.480374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:19.480646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:19.480911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:19.481525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:19.482522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:19.482522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:52:19.482522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:52:19.483758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:19.524763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 743/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:52:19 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:52:19.528768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:19.530274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:19.590675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4d5c9-98cf-4e5c-bf79-9f5f9d36b410 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:19.675495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:19.686637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:19.687246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:19.687520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:19.687818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:19.688448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:19.689457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:19.689457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:52:19.689457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:52:19.690330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:19.702529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 756/1499] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:52:19.706569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:19.708180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:19.719947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6131d7e-64cc-40a2-bbc6-f249922ee54b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:19.745379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:19.815277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:19.815946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:19.816222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:19.816490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:19.817155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:19.818179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:19.818179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:52:19.818179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:52:19.819068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a19e229-535d-4113-8143-dec272da269b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:19.831365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 744/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:52:19.835977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:19.838730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:19.877202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e08fbe-5708-4459-bfe0-60826a5accea admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:19.968657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:19.969346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:19.969627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:19.969899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] PATH_INFO: `/v3/projects/cbe0f436cf7b4a84bd829d3e5fa3186f/users/7173644cc7834bf2a154adcf7c5482b3/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:19.992891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe0f436cf7b4a84bd829d3e5fa3186f, user_id=7173644cc7834bf2a154adcf7c5482b3, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:19.993924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:19.993924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:52:19.993924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:52:19.995915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c67210-b270-4d5e-9589-0c9d303ba46a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:20.019381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 757/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:52:19 2022] PUT /identity/v3/projects/cbe0f436cf7b4a84bd829d3e5fa3186f/users/7173644cc7834bf2a154adcf7c5482b3/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:52:20.031491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328ea5-4a12-40c7-8241-bd8dc35dd88b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:20.031823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328ea5-4a12-40c7-8241-bd8dc35dd88b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:20.032104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65328ea5-4a12-40c7-8241-bd8dc35dd88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:20.087877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-65328ea5-4a12-40c7-8241-bd8dc35dd88b None None] MFA Rules not processed for user `7173644cc7834bf2a154adcf7c5482b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:52:20.089019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-65328ea5-4a12-40c7-8241-bd8dc35dd88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:20.168683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 745/1502] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:52:20.182807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dcefd-ca92-4419-a638-1738a08f699f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:20.183095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dcefd-ca92-4419-a638-1738a08f699f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:20.183371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8dcefd-ca92-4419-a638-1738a08f699f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:20.184787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 758/1503] 10.222.0.17 () {54 vars in 964 bytes} [Wed Sep 7 15:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:52:20.189526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052b78-2241-48a6-9d66-e687cf0b07d1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:20.189812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052b78-2241-48a6-9d66-e687cf0b07d1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:20.190086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052b78-2241-48a6-9d66-e687cf0b07d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:20.221384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:20.265722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-73052b78-2241-48a6-9d66-e687cf0b07d1 None None] MFA Rules not processed for user `396fc6e85a65437898bc9d964e429de3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:52:20.267017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-73052b78-2241-48a6-9d66-e687cf0b07d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:20.345603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 746/1504] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Sep 7 15:52:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:52:20.354176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dcefd-ca92-4419-a638-1738a08f699f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:20.355759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2c8dcefd-ca92-4419-a638-1738a08f699f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:20.404904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:20.460704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8dcefd-ca92-4419-a638-1738a08f699f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:20.539958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:20.540565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:20.540840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:20.541108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:20.542691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:20.638122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:20.639473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:20.642935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:20.658450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 759/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:52:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:52:20.702711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8ddfc-20e9-4b4d-add3-51bd6bda5b0a None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:20.702997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8ddfc-20e9-4b4d-add3-51bd6bda5b0a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:20.703276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d8ddfc-20e9-4b4d-add3-51bd6bda5b0a None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:20.704601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 747/1506] 10.222.0.17 () {52 vars in 830 bytes} [Wed Sep 7 15:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:52:20.708243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:20.710063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:20.810470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8950f-5700-45a7-8cf6-52855e18dccf service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:20.905458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:20.906078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:20.906260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:20.906528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] PATH_INFO: `/v3/users/7173644cc7834bf2a154adcf7c5482b3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:20.918290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7173644cc7834bf2a154adcf7c5482b3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:20.919311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:20.919311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:52:20.919311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:52:20.920542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:20.930623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:20.985707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7173644cc7834bf2a154adcf7c5482b3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:20.995574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7173644cc7834bf2a154adcf7c5482b3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:21.017877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7173644cc7834bf2a154adcf7c5482b3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:21.018400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7173644cc7834bf2a154adcf7c5482b3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:21.018814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7173644cc7834bf2a154adcf7c5482b3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:21.038836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 760/1507] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:52:20 2022] DELETE /identity/v3/users/7173644cc7834bf2a154adcf7c5482b3 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:52:21.043166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:21.043633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:21.043729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:21.099054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:52:21.100297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:21.176162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:21.189439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 748/1508] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:52:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:52:21.198920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:21.200172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:21.298403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e870d-94d4-487b-ae81-200479c45383 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:21.352757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:21.374268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:21.374875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:21.375152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:21.375422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:21.377025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:21.483410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:21.484801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdac4b8-cb20-44da-82db-984cb08e709a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:21.503362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 761/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:52:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:52:21.821298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:52:21.822589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:52:21.834404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b0255-ef5c-45a1-9c24-1cbfe1ff64bf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:52:21.929892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:52:21.930519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:52:21.930792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:52:21.931058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] PATH_INFO: `/v3/projects/cbe0f436cf7b4a84bd829d3e5fa3186f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:52:21.940578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe0f436cf7b4a84bd829d3e5fa3186f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:52:21.941594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:52:21.941594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:52:21.941594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:52:21.942779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:52:21.999947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe0f436cf7b4a84bd829d3e5fa3186f', 'request_id': 'req-21503faf-7c5e-453c-8480-7444d70c88ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:22.009382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe0f436cf7b4a84bd829d3e5fa3186f', 'request_id': 'req-21503faf-7c5e-453c-8480-7444d70c88ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:22.015870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe0f436cf7b4a84bd829d3e5fa3186f', 'request_id': 'req-21503faf-7c5e-453c-8480-7444d70c88ef'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:22.045377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] The token cache is being invalidate because project cbe0f436cf7b4a84bd829d3e5fa3186f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:52:22.045984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:22.046093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-21503faf-7c5e-453c-8480-7444d70c88ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:52:22.047448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 749/1510] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:52:21 2022] DELETE /identity/v3/projects/cbe0f436cf7b4a84bd829d3e5fa3186f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:52:31.501245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=62 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:52:32.043454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:53:52.485909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4027-567a-4f5e-8ffe-62e367e6e75b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:52.486497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4027-567a-4f5e-8ffe-62e367e6e75b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:52.486497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4027-567a-4f5e-8ffe-62e367e6e75b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:52.488359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:53:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:53:52.493043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e618cb-8d86-4a7c-965e-2b76721bdb30 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:52.493467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e618cb-8d86-4a7c-965e-2b76721bdb30 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:52.493595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e618cb-8d86-4a7c-965e-2b76721bdb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:52.575599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-76e618cb-8d86-4a7c-965e-2b76721bdb30 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:53:52.576813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-76e618cb-8d86-4a7c-965e-2b76721bdb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:53:52.681755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:53:52.833209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4027-567a-4f5e-8ffe-62e367e6e75b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:53:52.835451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4a1e4027-567a-4f5e-8ffe-62e367e6e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:53:52.954022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e4027-567a-4f5e-8ffe-62e367e6e75b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:53:52.959920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d865a3e5-44ee-4920-9551-8e92e2113530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:53:52.960521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a3e5-44ee-4920-9551-8e92e2113530 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:52.960802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a3e5-44ee-4920-9551-8e92e2113530 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:52.961073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d865a3e5-44ee-4920-9551-8e92e2113530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:52.968990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865a3e5-44ee-4920-9551-8e92e2113530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:53:52.970571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d865a3e5-44ee-4920-9551-8e92e2113530 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:53:52.980586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 763/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:53:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:53:54.066368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95191b7-4196-4f5f-89bc-6b50431d1d6f None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:54.066962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95191b7-4196-4f5f-89bc-6b50431d1d6f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:54.066962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95191b7-4196-4f5f-89bc-6b50431d1d6f None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:54.068281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:53:54.074246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6314528-da86-4bcd-bc3b-7f97bf2bb4b5 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:54.074657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6314528-da86-4bcd-bc3b-7f97bf2bb4b5 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:54.074792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6314528-da86-4bcd-bc3b-7f97bf2bb4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:54.093927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f6314528-da86-4bcd-bc3b-7f97bf2bb4b5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:53:54.095110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f6314528-da86-4bcd-bc3b-7f97bf2bb4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:53:54.103627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:53:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:53:54.252321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95191b7-4196-4f5f-89bc-6b50431d1d6f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:53:54.267111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95191b7-4196-4f5f-89bc-6b50431d1d6f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:53:54.273101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019244d0-6c62-49cf-bfa7-5ae6245e545e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:53:54.273685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019244d0-6c62-49cf-bfa7-5ae6245e545e service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:54.273961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019244d0-6c62-49cf-bfa7-5ae6245e545e service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:54.274228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019244d0-6c62-49cf-bfa7-5ae6245e545e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:54.282261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019244d0-6c62-49cf-bfa7-5ae6245e545e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:53:54.283860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019244d0-6c62-49cf-bfa7-5ae6245e545e service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:53:54.292988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 752/1516] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:53:55.155305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f29e6-e0a9-421d-aeae-877035da7b78 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:55.155952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f29e6-e0a9-421d-aeae-877035da7b78 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:55.155952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f29e6-e0a9-421d-aeae-877035da7b78 None None] PATH_INFO: `/v3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:55.157185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 765/1517] 10.222.0.23 () {54 vars in 925 bytes} [Wed Sep 7 15:53:55 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:53:55.163773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b73f5-6818-4643-9f9b-c439cd75d913 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:53:55.164173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b73f5-6818-4643-9f9b-c439cd75d913 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:53:55.164333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811b73f5-6818-4643-9f9b-c439cd75d913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:53:55.223687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-811b73f5-6818-4643-9f9b-c439cd75d913 None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:53:55.224884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-811b73f5-6818-4643-9f9b-c439cd75d913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:53:55.241806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:53:55.295532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 753/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Sep 7 15:53:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:04.005045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec33e6-bb6f-4ca1-bd4e-4cd7a96d9c10 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:04.005604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec33e6-bb6f-4ca1-bd4e-4cd7a96d9c10 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:04.005604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ec33e6-bb6f-4ca1-bd4e-4cd7a96d9c10 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:04.006952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 766/1519] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 15:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:54:04.012128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bc9c3-cd12-42c6-9564-150a78a8c246 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:04.012584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bc9c3-cd12-42c6-9564-150a78a8c246 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:04.012702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399bc9c3-cd12-42c6-9564-150a78a8c246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:04.033252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-399bc9c3-cd12-42c6-9564-150a78a8c246 None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:04.035019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-399bc9c3-cd12-42c6-9564-150a78a8c246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:04.044000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 754/1520] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 15:54:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:04.052267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec33e6-bb6f-4ca1-bd4e-4cd7a96d9c10 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:04.065299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ec33e6-bb6f-4ca1-bd4e-4cd7a96d9c10 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:04.070141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524be53f-2388-4641-b7bb-a3940de1fab3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:04.070761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524be53f-2388-4641-b7bb-a3940de1fab3 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:04.071038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524be53f-2388-4641-b7bb-a3940de1fab3 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:04.071308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524be53f-2388-4641-b7bb-a3940de1fab3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:04.079205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524be53f-2388-4641-b7bb-a3940de1fab3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:04.080881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524be53f-2388-4641-b7bb-a3940de1fab3 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:04.090625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 767/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:05.770205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e717-2256-45fe-a584-a42e16a87844 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:05.770748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e717-2256-45fe-a584-a42e16a87844 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:05.770748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d939e717-2256-45fe-a584-a42e16a87844 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:05.772115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:54:05.778192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a74e7-bb91-48b8-8105-1b9861c3c145 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:05.778594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a74e7-bb91-48b8-8105-1b9861c3c145 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:05.778749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4a74e7-bb91-48b8-8105-1b9861c3c145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:05.799467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-eb4a74e7-bb91-48b8-8105-1b9861c3c145 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:05.800781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-eb4a74e7-bb91-48b8-8105-1b9861c3c145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:05.808438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:05.962949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e717-2256-45fe-a584-a42e16a87844 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:05.977577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d939e717-2256-45fe-a584-a42e16a87844 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:05.982977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7836-a124-4078-8072-9a8fab3abdb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:05.983583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7836-a124-4078-8072-9a8fab3abdb0 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:05.983865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7836-a124-4078-8072-9a8fab3abdb0 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:05.984140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7836-a124-4078-8072-9a8fab3abdb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:05.992237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7836-a124-4078-8072-9a8fab3abdb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:05.993809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7836-a124-4078-8072-9a8fab3abdb0 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:06.003122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 756/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:07.071774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e44c-b0d9-41d5-bf1e-917711bb1502 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:07.072372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e44c-b0d9-41d5-bf1e-917711bb1502 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:07.072372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1e44c-b0d9-41d5-bf1e-917711bb1502 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:07.073716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 7 15:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:54:07.080059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f304-7d9a-4dac-b4c7-5e54fa78dd4b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:07.080491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f304-7d9a-4dac-b4c7-5e54fa78dd4b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:07.080656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110f304-7d9a-4dac-b4c7-5e54fa78dd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:07.101021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c110f304-7d9a-4dac-b4c7-5e54fa78dd4b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:07.102592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c110f304-7d9a-4dac-b4c7-5e54fa78dd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:07.110930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 7 15:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:07.267434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1e44c-b0d9-41d5-bf1e-917711bb1502 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:07.282564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1e44c-b0d9-41d5-bf1e-917711bb1502 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:07.288612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4de0d0-3360-465a-830c-0c5e45b2507b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:07.289212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4de0d0-3360-465a-830c-0c5e45b2507b service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:07.289499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4de0d0-3360-465a-830c-0c5e45b2507b service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:07.289772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4de0d0-3360-465a-830c-0c5e45b2507b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:07.297870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4de0d0-3360-465a-830c-0c5e45b2507b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:07.299450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4de0d0-3360-465a-830c-0c5e45b2507b service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:07.308805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 770/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:11.390226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce321f7-3aff-4bfd-87be-1f5f388bd467 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:11.390780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce321f7-3aff-4bfd-87be-1f5f388bd467 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:11.390780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce321f7-3aff-4bfd-87be-1f5f388bd467 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:11.392308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:54:11.397450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf7a2-e72d-4bc8-889b-8d84ef30b19d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:11.397911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf7a2-e72d-4bc8-889b-8d84ef30b19d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:11.398012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfbf7a2-e72d-4bc8-889b-8d84ef30b19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:11.417732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3bfbf7a2-e72d-4bc8-889b-8d84ef30b19d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:11.418855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3bfbf7a2-e72d-4bc8-889b-8d84ef30b19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:11.427279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:11.713987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce321f7-3aff-4bfd-87be-1f5f388bd467 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:11.715726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3ce321f7-3aff-4bfd-87be-1f5f388bd467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:11.795136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce321f7-3aff-4bfd-87be-1f5f388bd467 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:11.798708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46195c5-af60-435f-bcfd-d48822f008c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:11.799304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46195c5-af60-435f-bcfd-d48822f008c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:11.799606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46195c5-af60-435f-bcfd-d48822f008c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:11.799882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46195c5-af60-435f-bcfd-d48822f008c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:11.808689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46195c5-af60-435f-bcfd-d48822f008c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:11.810103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46195c5-af60-435f-bcfd-d48822f008c2 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:11.819358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 759/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:13.955048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a0397-dfd2-4f54-abd4-f890a9a5297c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:13.955634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a0397-dfd2-4f54-abd4-f890a9a5297c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:13.955634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a0397-dfd2-4f54-abd4-f890a9a5297c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:13.957032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:54:13.962313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e6ee3-708b-4777-a436-4a51eca218f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:13.962763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e6ee3-708b-4777-a436-4a51eca218f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:13.962873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e6ee3-708b-4777-a436-4a51eca218f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:13.984269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fc4e6ee3-708b-4777-a436-4a51eca218f5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:13.986054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc4e6ee3-708b-4777-a436-4a51eca218f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:13.995870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:54:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:14.286436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a0397-dfd2-4f54-abd4-f890a9a5297c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:14.296987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a0397-dfd2-4f54-abd4-f890a9a5297c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:14.300503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f475f25-2163-48eb-b206-0f803e67b249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:14.301574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475f25-2163-48eb-b206-0f803e67b249 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:14.301847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475f25-2163-48eb-b206-0f803e67b249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:14.302113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f475f25-2163-48eb-b206-0f803e67b249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:14.310628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475f25-2163-48eb-b206-0f803e67b249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:14.312082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f475f25-2163-48eb-b206-0f803e67b249 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:14.321121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 773/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:18.126827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37742-1c64-44e2-8ffc-00c94e01cd57 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:18.127366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37742-1c64-44e2-8ffc-00c94e01cd57 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:18.127366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df37742-1c64-44e2-8ffc-00c94e01cd57 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:18.129311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:54:18.134476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f49c2-3e51-423a-81f2-0ee34aadbd63 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:18.134963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f49c2-3e51-423a-81f2-0ee34aadbd63 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:18.135069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f49c2-3e51-423a-81f2-0ee34aadbd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:18.154899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c17f49c2-3e51-423a-81f2-0ee34aadbd63 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:18.156166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c17f49c2-3e51-423a-81f2-0ee34aadbd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:18.163986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:54:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:18.466529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37742-1c64-44e2-8ffc-00c94e01cd57 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:18.468382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0df37742-1c64-44e2-8ffc-00c94e01cd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:18.532603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:54:18.546410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df37742-1c64-44e2-8ffc-00c94e01cd57 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:18.550007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e4809-52ca-494a-91cf-b0298b20f33a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:18.551113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e4809-52ca-494a-91cf-b0298b20f33a service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:18.551390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e4809-52ca-494a-91cf-b0298b20f33a service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:18.551691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e4809-52ca-494a-91cf-b0298b20f33a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:18.560458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e4809-52ca-494a-91cf-b0298b20f33a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:18.561863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e4809-52ca-494a-91cf-b0298b20f33a service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:18.571903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 762/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:20.705318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413703fd-520a-4ee5-8ecb-f0ea0a82d2c3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:20.705891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413703fd-520a-4ee5-8ecb-f0ea0a82d2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:20.705891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413703fd-520a-4ee5-8ecb-f0ea0a82d2c3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:20.707795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 7 15:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:54:20.713534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432718b6-33eb-4d3f-90b2-816cb15dd481 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:20.714030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432718b6-33eb-4d3f-90b2-816cb15dd481 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:20.714030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432718b6-33eb-4d3f-90b2-816cb15dd481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:20.734208 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-432718b6-33eb-4d3f-90b2-816cb15dd481 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:54:20.735420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-432718b6-33eb-4d3f-90b2-816cb15dd481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:20.743066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 7 15:54:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:54:21.036825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413703fd-520a-4ee5-8ecb-f0ea0a82d2c3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:54:21.038521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-413703fd-520a-4ee5-8ecb-f0ea0a82d2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:54:21.051952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413703fd-520a-4ee5-8ecb-f0ea0a82d2c3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:54:21.056806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdf8f18-8312-40e5-93ba-d319fae740ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:54:21.057748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8f18-8312-40e5-93ba-d319fae740ce service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:54:21.058033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8f18-8312-40e5-93ba-d319fae740ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:54:21.058304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdf8f18-8312-40e5-93ba-d319fae740ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:54:21.068023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf8f18-8312-40e5-93ba-d319fae740ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:54:21.069449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdf8f18-8312-40e5-93ba-d319fae740ce service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:54:21.078789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 776/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:54:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:54:30.729345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=11 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:54:31.074747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:47.001125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.001831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.001831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.029015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.030259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.046649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 764/1540] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:47.055622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.056786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.057295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.083226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.084844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.095625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 777/1541] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.100743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.111415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1fb6d-bdf3-4650-b201-b7483fe68e2d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.121419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.122284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.122569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.122844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.123505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:47.123834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.125582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.145684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 765/1542] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:47.154039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.168858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c03d3-1819-4e88-b940-71a268a85b23 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.179669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738771b6-0485-4cd0-81d7-db5047f6f144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.180620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738771b6-0485-4cd0-81d7-db5047f6f144 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.181146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738771b6-0485-4cd0-81d7-db5047f6f144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.181643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738771b6-0485-4cd0-81d7-db5047f6f144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.182493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738771b6-0485-4cd0-81d7-db5047f6f144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.187566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738771b6-0485-4cd0-81d7-db5047f6f144 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.247370 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 778/1543] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/projects => generated 363 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:47.251648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.266334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72eb37-44d2-44a2-ada1-8875304aa451 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.341891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.342514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.342797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.343072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.344016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.346210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.380641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.381310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 766/1544] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/users => generated 282 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:47.381551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.382190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.387895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.404368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e0526-32fa-4128-ab05-82b6d18bb561 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.419188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.420221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338e13de-b54f-49a5-a060-9df6495221db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.420868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e13de-b54f-49a5-a060-9df6495221db admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.421313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e13de-b54f-49a5-a060-9df6495221db admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.421596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338e13de-b54f-49a5-a060-9df6495221db admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.421955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.422326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e13de-b54f-49a5-a060-9df6495221db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.427573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338e13de-b54f-49a5-a060-9df6495221db admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.439214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 779/1545] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:47.442425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 767/1546] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:55:47.447981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.458149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a8c1c-b95a-4a50-98bb-73fc14ee52c5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.458513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a8c1c-b95a-4a50-98bb-73fc14ee52c5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.458766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518a8c1c-b95a-4a50-98bb-73fc14ee52c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.468560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31efb09e-e0e8-4c6a-808d-6062cae0ff49 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.480875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.482112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.482626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.483121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.484046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.486284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.496014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-518a8c1c-b95a-4a50-98bb-73fc14ee52c5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.497424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-518a8c1c-b95a-4a50-98bb-73fc14ee52c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.502290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 780/1547] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:47.507324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.529727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d308f2-f22f-4293-8669-97adeedf5d5d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.530882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 768/1548] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.534178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.534178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.534178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.555063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.556221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.556748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.557259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] PATH_INFO: `/v3/projects/755d15a7feea47eea8bfaaa4da4c4a2e/users/77b0a15642854df3a8131900cd0b1ee2/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.564135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.566344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.583450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 769/1549] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.584161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.603111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c934bc-da3d-4ba4-804d-b251f3f2c2bd None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.605091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755d15a7feea47eea8bfaaa4da4c4a2e, user_id=77b0a15642854df3a8131900cd0b1ee2, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.607800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.608362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.608477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.608695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.609183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.610830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:47.610830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.619894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4620f47-4c80-4aa9-b342-8ebdcaebabb3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.638783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 770/1550] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:47.640031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e672cb3-b175-43b2-9674-7ce6af674bef None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.640323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e672cb3-b175-43b2-9674-7ce6af674bef None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.640601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e672cb3-b175-43b2-9674-7ce6af674bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.642219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 781/1551] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:47 2022] PUT /identity/v3/projects/755d15a7feea47eea8bfaaa4da4c4a2e/users/77b0a15642854df3a8131900cd0b1ee2/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:47.650646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.680065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1e672cb3-b175-43b2-9674-7ce6af674bef None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.681310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e672cb3-b175-43b2-9674-7ce6af674bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.739887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7dc79b-3729-40ed-af46-4b1432153b99 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.756356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f8bd8b-4e66-491a-aa92-f9dd5dfe3e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.757359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8bd8b-4e66-491a-aa92-f9dd5dfe3e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.757938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8bd8b-4e66-491a-aa92-f9dd5dfe3e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.758446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f8bd8b-4e66-491a-aa92-f9dd5dfe3e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.759429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8bd8b-4e66-491a-aa92-f9dd5dfe3e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.761640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f8bd8b-4e66-491a-aa92-f9dd5dfe3e01 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.787995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 771/1552] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.793512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.793797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.794063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.799365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 782/1553] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/projects => generated 383 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:55:47.800980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be3d3f-4c99-428f-9e39-49af90296682 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.805006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be3d3f-4c99-428f-9e39-49af90296682 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.806162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be3d3f-4c99-428f-9e39-49af90296682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.825044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] MFA Rules not processed for user `77b0a15642854df3a8131900cd0b1ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.826257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.827009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a0be3d3f-4c99-428f-9e39-49af90296682 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.828225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a0be3d3f-4c99-428f-9e39-49af90296682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.846117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:47.889485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 783/1554] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.890595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc369c8-d213-4e20-b2d6-f580652d19fd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.890884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc369c8-d213-4e20-b2d6-f580652d19fd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.891158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc369c8-d213-4e20-b2d6-f580652d19fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.894197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 772/1555] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.895843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.906486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d272cc-2d6b-4d66-911d-21317ac6e16d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.909872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:47.911330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.911961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.912243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.912513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.913158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:47.913454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.915290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.915905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6dc369c8-d213-4e20-b2d6-f580652d19fd None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.917556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6dc369c8-d213-4e20-b2d6-f580652d19fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.925576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 784/1556] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.927692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 773/1557] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:47.928451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:47.928913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b37956-eb5c-4371-b188-bd98cd74f5cc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.929269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b37956-eb5c-4371-b188-bd98cd74f5cc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.929614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b37956-eb5c-4371-b188-bd98cd74f5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.938326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a4bd19-55f0-4700-bc40-2fdb153a9d1f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:47.943024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e22366-1445-4273-95b4-db448813a205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:47.943641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22366-1445-4273-95b4-db448813a205 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.943919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22366-1445-4273-95b4-db448813a205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.944189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e22366-1445-4273-95b4-db448813a205 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.944868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e22366-1445-4273-95b4-db448813a205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:47.946756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e22366-1445-4273-95b4-db448813a205 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:47.950270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e5b37956-eb5c-4371-b188-bd98cd74f5cc None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.951450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e5b37956-eb5c-4371-b188-bd98cd74f5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.960587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 785/1558] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.962592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f41d27-7855-430c-b504-887f85f73e82 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.962875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f41d27-7855-430c-b504-887f85f73e82 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.963163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f41d27-7855-430c-b504-887f85f73e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.971364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 774/1559] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:47.972839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.973160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.973437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:47.986346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-60f41d27-7855-430c-b504-887f85f73e82 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.987555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-60f41d27-7855-430c-b504-887f85f73e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.992938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:47.994122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:47.995880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 786/1560] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:47.997133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d9e1-3eab-4e77-addf-480e610ddc49 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:47.997420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d9e1-3eab-4e77-addf-480e610ddc49 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:47.997697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d9e1-3eab-4e77-addf-480e610ddc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.007454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 775/1561] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:48.008868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.021389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-09b7d9e1-3eab-4e77-addf-480e610ddc49 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:48.022583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09b7d9e1-3eab-4e77-addf-480e610ddc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:48.024819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b3b7d-f3b9-41b6-b252-bc9201ec16a7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.029303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.029890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.030166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.030397 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 787/1562] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:48.030511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.031046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.031650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6ea56-a128-491d-a894-dc1b9f4d08f7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.031929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6ea56-a128-491d-a894-dc1b9f4d08f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.032109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:55:48.032109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:55:48.032109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:55:48.032352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6ea56-a128-491d-a894-dc1b9f4d08f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.033301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.051760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4ab6ea56-a128-491d-a894-dc1b9f4d08f7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:48.053223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4ab6ea56-a128-491d-a894-dc1b9f4d08f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:48.061948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 788/1563] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:48.062533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 776/1564] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.063048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.063935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.064222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.064593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.084445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:48.085635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:48.128604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c9a3c-7126-4967-91f9-26848188122f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.133561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aace5a1-554f-4803-bfb4-0a6d2ec65c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.134171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aace5a1-554f-4803-bfb4-0a6d2ec65c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.134454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aace5a1-554f-4803-bfb4-0a6d2ec65c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.134726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aace5a1-554f-4803-bfb4-0a6d2ec65c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.135374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aace5a1-554f-4803-bfb4-0a6d2ec65c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.137420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aace5a1-554f-4803-bfb4-0a6d2ec65c2f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.145343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 789/1565] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:48.148408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.159866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fff398-a532-49ec-87c4-d8915dd5607d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.164498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f795b097-2f5d-419b-b51e-b842cfc7cc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.165116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795b097-2f5d-419b-b51e-b842cfc7cc51 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.165397 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795b097-2f5d-419b-b51e-b842cfc7cc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.165672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f795b097-2f5d-419b-b51e-b842cfc7cc51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.166345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795b097-2f5d-419b-b51e-b842cfc7cc51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.168001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 777/1566] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.168442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f795b097-2f5d-419b-b51e-b842cfc7cc51 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.169241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.170963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.170963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.183134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 790/1567] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:48.183134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08574623-490a-4909-b884-f0f459ad7588 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.183134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08574623-490a-4909-b884-f0f459ad7588 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.183134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08574623-490a-4909-b884-f0f459ad7588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.190008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:48.191233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:48.201636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-08574623-490a-4909-b884-f0f459ad7588 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:48.202805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-08574623-490a-4909-b884-f0f459ad7588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:48.235265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:48.251869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 778/1568] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:48.252689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.261766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af622aba-28fc-4ca6-b17d-ad8440a6efe0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.262659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 791/1569] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:48.263467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08574623-490a-4909-b884-f0f459ad7588 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.266345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.266938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.267222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.267494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.268171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:48.268478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.270112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.273162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08574623-490a-4909-b884-f0f459ad7588 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.277924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.278648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.278936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.279207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.280013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.281028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:55:48.281028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:55:48.281028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:55:48.289083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 779/1570] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:48.289083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.289323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.292479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0001faee-c05b-460f-a63c-a37910fecc61 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.297251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.297845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.298127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.298400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.299035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:48.299343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.300999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.308160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 792/1571] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.308160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.321604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 780/1572] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:48.321604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.322211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878eb61-c2e4-4ce7-927e-37cfd8d5924e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.325820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.326440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.326717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.326987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.327673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:48.327981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.329031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:55:48.329031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:55:48.329031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:55:48.330016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.333145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28fe5d-c205-4fc3-adb0-d7d08960a287 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.338394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.338987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.339274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.340029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.340665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.342575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.342699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 793/1573] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:48.343449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.355592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f030e9e-75b8-4961-aa04-95ab29c51655 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.361131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.361740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.362019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.362319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.362952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.364768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.365870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 781/1574] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/users => generated 294 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.366693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.376374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c041f1a1-604b-42db-9a38-09a175deb349 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.378846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 794/1575] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:48.380088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.384588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.385181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.385464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.385736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.386379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:48.386699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.388728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.390050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18bf6ee-1702-4126-802a-5df7b1f15eb8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.394728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.395413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.395800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.396143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.396837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:48.397212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.398880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.401477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 782/1576] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:48.402320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.412599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf28c4-97f3-49e3-88ad-33284d0abbd8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.414817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 795/1577] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:48.415662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.424387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.424981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.425257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.425527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.426706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.427210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a151a5-a653-419b-85c8-841686006ed7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.428684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.431765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.432342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.432620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.432891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.433523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.435085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.437693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:48.451599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 796/1578] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:48.451599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.456592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 783/1579] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.457418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.460027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e472d35-6e33-4c2d-9d65-0ba65e36e3a4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.517191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.517647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d695bb-3e56-4be4-b741-fe9ae0c4734b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.517773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.518038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.518329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.518956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.520940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.522245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.522843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.523117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.523387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.524036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.525951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.547714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 797/1580] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.548627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.561436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 784/1581] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.561436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.613498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684ffec-a890-49fc-9d57-7baca57faaef admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.619333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:48.624568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf31f19a-ae4c-4c51-8d4f-0e2e65168e01 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.630543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.631504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.631905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.632635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] PATH_INFO: `/v3/projects/d1416faa95b24d76b2189a0cf79e3389/users/53642660b6584d5e89f8e3e9c0d3a301/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.647503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1416faa95b24d76b2189a0cf79e3389, user_id=53642660b6584d5e89f8e3e9c0d3a301, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.648590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:55:48.648590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:55:48.648590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:55:48.650537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.665231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 785/1582] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:48 2022] PUT /identity/v3/projects/d1416faa95b24d76b2189a0cf79e3389/users/53642660b6584d5e89f8e3e9c0d3a301/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:48.666010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.682772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.683419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.683732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.684116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.684766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.686412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.696364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 798/1583] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:48.700180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.710103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981c816-8ce9-4d0e-8561-7fbce31379f8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.728423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eda7eb-6487-44b9-9032-0f205b982db5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.733049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.733760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.733975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:48.734126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.734363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.734984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.737311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.766856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 786/1584] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.766856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.774540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.775125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.775520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.775831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.776468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.778456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.805462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 799/1585] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.806244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.829406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c18331-a38d-4adf-b1a0-0e9daf398191 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.868223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d03c08-5158-478d-8845-7596a3bdbae3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.872809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.873440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.873715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.873984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.874599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.876550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.886139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.886794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.887071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.887337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] PATH_INFO: `/v3/projects/0fbad6f2a321449395f652e6f8fe9bd0/users/ec54877e04804aedafc3a49cef81511d/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.898459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 800/1586] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.899056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.905120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbad6f2a321449395f652e6f8fe9bd0, user_id=ec54877e04804aedafc3a49cef81511d, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.907842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.908075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a733f-a0de-403c-89e4-ec7d565d3a41 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.910052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:48.913371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfad6036-e286-4bd3-a2a1-eb8d33bdd98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.914031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad6036-e286-4bd3-a2a1-eb8d33bdd98b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.914352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad6036-e286-4bd3-a2a1-eb8d33bdd98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.914626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfad6036-e286-4bd3-a2a1-eb8d33bdd98b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.915295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad6036-e286-4bd3-a2a1-eb8d33bdd98b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.917350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfad6036-e286-4bd3-a2a1-eb8d33bdd98b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:48.922314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 787/1587] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:48 2022] PUT /identity/v3/projects/0fbad6f2a321449395f652e6f8fe9bd0/users/ec54877e04804aedafc3a49cef81511d/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:48.923094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:48.939250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 801/1588] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/users => generated 302 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:48.940632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.940916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.941188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.968017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] MFA Rules not processed for user `53642660b6584d5e89f8e3e9c0d3a301`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:48.969214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:48.989275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ef32f-7e59-409f-8d44-f3289c2b55e1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:48.993894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:48.994494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:48.994774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:48.995045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:48.995760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:48.997682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.019467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 788/1589] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:49.020464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.029968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd9600-9af3-4e76-9d36-01753e44f578 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.039133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 802/1590] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:49.039352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.039491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.039620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.039620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.039620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.039620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.039620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.057873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 789/1591] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.058645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.059701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04816f51-ad41-4079-a266-cd11c418fde4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.064592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.065115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.065895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.065895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.066557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.068764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.070551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387e5086-3e61-4cf7-a97a-7782e08fbed6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.075743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.076461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.076845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.077253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.077988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.080235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.081331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:49.094047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 803/1592] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:49.095404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.105016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 790/1593] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:49.106042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.106489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fce1df3-9c98-4375-aaf1-ec696ea6d3d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.111111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.111817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.112122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.112369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.113031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.115184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.116582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1372e-c54b-4c7a-bcca-78154cb579af admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.121023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.121711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.121992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.122261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.122928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.124466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.126229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 804/1594] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.127196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.137595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31944733-a3c7-4ad4-bba0-1627f8c58ac5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.139823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 791/1595] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.140761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.144550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.145200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.145478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.145749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.146376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.148010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.150397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7f62c7-da11-41d6-89b4-672c1b9728e5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.155638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ecdbe5-4095-4642-ab72-9daacbfbb316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.155802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecdbe5-4095-4642-ab72-9daacbfbb316 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.157357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecdbe5-4095-4642-ab72-9daacbfbb316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.157357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ecdbe5-4095-4642-ab72-9daacbfbb316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.157357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecdbe5-4095-4642-ab72-9daacbfbb316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.158365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 805/1596] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.158486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ecdbe5-4095-4642-ab72-9daacbfbb316 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.159281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.171037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e22c0-9e09-43a0-9947-642d46446f27 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.172047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 792/1597] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.175191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4dc42-c2de-464a-b14c-178de60d8fdf None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.175191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4dc42-c2de-464a-b14c-178de60d8fdf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.175191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4dc42-c2de-464a-b14c-178de60d8fdf None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.175525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 793/1598] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 15:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:55:49.177304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.177958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.178234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.178504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] PATH_INFO: `/v3/projects/64aae5509a124aebbb92ff5dc82e90cf/users/d7e96593e2814e8b9f7c0f320aa17c3c/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.180497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4dc42-c2de-464a-b14c-178de60d8fdf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.192979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4dc42-c2de-464a-b14c-178de60d8fdf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.193336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64aae5509a124aebbb92ff5dc82e90cf, user_id=d7e96593e2814e8b9f7c0f320aa17c3c, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.196011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.197506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.198085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.198365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.198642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.199277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.200906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.210056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 806/1599] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/64aae5509a124aebbb92ff5dc82e90cf/users/d7e96593e2814e8b9f7c0f320aa17c3c/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.210845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.215383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 794/1600] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.215383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.224710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac294f21-9de0-4f45-92df-3cd9df90cbcf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.274597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372313e-b792-402e-a096-bd0434122e10 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.280355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.280989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.281273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.281552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.282203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.283850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.288478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.289087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.289368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.290082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.290729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.292392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.294405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 807/1601] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.295253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.303121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 795/1602] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.304148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.308514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489409d-a039-4cf4-825c-9f8c2c61f9a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.313305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbb5b2-dd96-4043-96e9-5f09ff1284d5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.314094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8dcfcc-79d5-469e-8cba-5f5819c33f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.314752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dcfcc-79d5-469e-8cba-5f5819c33f41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.315036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dcfcc-79d5-469e-8cba-5f5819c33f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.315315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8dcfcc-79d5-469e-8cba-5f5819c33f41 admin admin] PATH_INFO: `/v3/projects/0fbad6f2a321449395f652e6f8fe9bd0/users/ec54877e04804aedafc3a49cef81511d/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.317376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8dcfcc-79d5-469e-8cba-5f5819c33f41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fbad6f2a321449395f652e6f8fe9bd0, user_id=ec54877e04804aedafc3a49cef81511d, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.317877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.318443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.318718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.318986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.319623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.320063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8dcfcc-79d5-469e-8cba-5f5819c33f41 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.321165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.331130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 796/1603] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.331523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 808/1604] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/0fbad6f2a321449395f652e6f8fe9bd0/users/ec54877e04804aedafc3a49cef81511d/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.331823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.332806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.333095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.333377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.348474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8a4c7-c88b-477b-8741-66f0b7b06bf3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.352489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:49.373451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:49.374670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:49.407250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5d8d4-43eb-40ff-8477-0285b84bc6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.407886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d8d4-43eb-40ff-8477-0285b84bc6da admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.408164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d8d4-43eb-40ff-8477-0285b84bc6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.408433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5d8d4-43eb-40ff-8477-0285b84bc6da admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.409066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5d8d4-43eb-40ff-8477-0285b84bc6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.411193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5d8d4-43eb-40ff-8477-0285b84bc6da admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.420806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 797/1605] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.421981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.422262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.422538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.446476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:49.446693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 809/1606] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:49.447436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.447718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:49.455158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 798/1607] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:49.457930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.458044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400767e-609c-4e4c-b661-a17c91c6d1fc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.462911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.463531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.463859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.464133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.464829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.466456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.467009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11af187b-2d3d-4790-adcb-f46569db55f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.471513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.472132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.472404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.472611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:49.472719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.473864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.475432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.478930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 810/1608] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.480167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.487811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 799/1609] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:49.488506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.493674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67193893-f7b9-4497-83f9-c5687bb7ea74 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.498146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bc230-8761-43ed-a7b3-210a11b76ab7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.498285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4828b9f-b3ca-41ab-bce9-f31295ef66fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.498981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828b9f-b3ca-41ab-bce9-f31295ef66fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.499264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828b9f-b3ca-41ab-bce9-f31295ef66fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.499613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4828b9f-b3ca-41ab-bce9-f31295ef66fd admin admin] PATH_INFO: `/v3/projects/5d24ce688df84f049775fa082eefcdce/users/de43a221bd8148b39eabc978b97f0e2a/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.502920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.503611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.503852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.504170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] PATH_INFO: `/v3/projects/5d824a9763c149ff8e223edc7f13ed6a/users/0d85459b723443ed8663854e03f3103d/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.514751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828b9f-b3ca-41ab-bce9-f31295ef66fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d24ce688df84f049775fa082eefcdce, user_id=de43a221bd8148b39eabc978b97f0e2a, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.517471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4828b9f-b3ca-41ab-bce9-f31295ef66fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.518514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d824a9763c149ff8e223edc7f13ed6a, user_id=0d85459b723443ed8663854e03f3103d, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.521791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.531031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 811/1610] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/5d24ce688df84f049775fa082eefcdce/users/de43a221bd8148b39eabc978b97f0e2a/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.532278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.532577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.532842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.535985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 800/1611] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/5d824a9763c149ff8e223edc7f13ed6a/users/0d85459b723443ed8663854e03f3103d/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.536699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.562292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] MFA Rules not processed for user `ec54877e04804aedafc3a49cef81511d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:49.563504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:49.597730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01afabd0-4a00-4569-b009-5287b1f9417f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.603798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.604460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.604760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.604999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] PATH_INFO: `/v3/projects/de82c2f32eb4464a9a7cb139ab8a1b76/users/ba225ed230cf48a79d5e5acc424b157e/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.620614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de82c2f32eb4464a9a7cb139ab8a1b76, user_id=ba225ed230cf48a79d5e5acc424b157e, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.623246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.637584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 801/1612] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/de82c2f32eb4464a9a7cb139ab8a1b76/users/ba225ed230cf48a79d5e5acc424b157e/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.638565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.644319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 812/1613] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:49.645807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.694535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:49.704205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972a9e3-5c19-4f69-ad0c-19f8b423e966 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.725164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acbb468-1090-4e0f-9d64-a92deb091aa3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.730924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.731688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.731972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.732242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.755726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.756343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.756628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.756905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.787839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:49.789674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.791265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.800330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 802/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:49.801137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.824497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.826084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.836620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 813/1615] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:49.847494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:49.857679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716a2339-53bb-414f-89d0-8c24b1d532ad service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.863178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d588700-a08a-44e8-82cd-8911448f2556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.864774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d588700-a08a-44e8-82cd-8911448f2556 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.865286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d588700-a08a-44e8-82cd-8911448f2556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.865566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d588700-a08a-44e8-82cd-8911448f2556 admin admin] PATH_INFO: `/v3/projects/20596aac52254a159269aa634116f080/users/7dc84ece6eb84315b54defd89be8a95c/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.870219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a89e23-73e2-4a61-b292-92175eb74c05 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:49.875892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60da6c79-ff9a-4c9d-a909-340caf23f0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:49.876707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6c79-ff9a-4c9d-a909-340caf23f0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.877112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6c79-ff9a-4c9d-a909-340caf23f0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.877485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da6c79-ff9a-4c9d-a909-340caf23f0f2 admin admin] PATH_INFO: `/v3/projects/2d357053af334b5090309eae0d444d95/users/4feedcde86d04d2395b09ae65dee4ad9/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.883145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d588700-a08a-44e8-82cd-8911448f2556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20596aac52254a159269aa634116f080, user_id=7dc84ece6eb84315b54defd89be8a95c, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.886267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d588700-a08a-44e8-82cd-8911448f2556 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.895020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da6c79-ff9a-4c9d-a909-340caf23f0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d357053af334b5090309eae0d444d95, user_id=4feedcde86d04d2395b09ae65dee4ad9, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:49.898029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60da6c79-ff9a-4c9d-a909-340caf23f0f2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:49.901924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 814/1616] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/20596aac52254a159269aa634116f080/users/7dc84ece6eb84315b54defd89be8a95c/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.904896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f3e5b-75dc-49f5-ae8a-3e0a1ff0d89c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.904896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f3e5b-75dc-49f5-ae8a-3e0a1ff0d89c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.904896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f3e5b-75dc-49f5-ae8a-3e0a1ff0d89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.917539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 803/1617] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:49 2022] PUT /identity/v3/projects/2d357053af334b5090309eae0d444d95/users/4feedcde86d04d2395b09ae65dee4ad9/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:49.925154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c32f3e5b-75dc-49f5-ae8a-3e0a1ff0d89c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:49.926454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c32f3e5b-75dc-49f5-ae8a-3e0a1ff0d89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:49.930879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.931242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.931831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:49.965679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] MFA Rules not processed for user `0d85459b723443ed8663854e03f3103d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:49.967118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:49.998942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 815/1618] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:49.998942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9051-9c4f-40dd-99d8-17c3b532716e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:49.998942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9051-9c4f-40dd-99d8-17c3b532716e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:49.998942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d9051-9c4f-40dd-99d8-17c3b532716e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.033825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8f3d9051-9c4f-40dd-99d8-17c3b532716e None None] MFA Rules not processed for user `ba225ed230cf48a79d5e5acc424b157e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:50.034887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8f3d9051-9c4f-40dd-99d8-17c3b532716e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:50.045619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:50.063085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 804/1619] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:50.063085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.114444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 816/1620] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:50.116129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8144bb-24ee-4209-a8f2-3792289ca30a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.116770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8144bb-24ee-4209-a8f2-3792289ca30a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.117051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8144bb-24ee-4209-a8f2-3792289ca30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.128791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f33704-4b9e-415e-a226-b9ab9bb9b601 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.132868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.133483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.133798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.134018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.141641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ee8144bb-24ee-4209-a8f2-3792289ca30a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:50.143340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee8144bb-24ee-4209-a8f2-3792289ca30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:50.156040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 817/1621] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:50.157629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945df6f-8f59-4860-8c5c-e4c0d1319578 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.157983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945df6f-8f59-4860-8c5c-e4c0d1319578 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.158517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945df6f-8f59-4860-8c5c-e4c0d1319578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.195990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a945df6f-8f59-4860-8c5c-e4c0d1319578 None None] MFA Rules not processed for user `4feedcde86d04d2395b09ae65dee4ad9`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:50.197307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a945df6f-8f59-4860-8c5c-e4c0d1319578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:50.210418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.211174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:50.212315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.223800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 805/1622] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:50.224434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.226266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:50.317897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17af3bd5-0985-483b-8243-c2979fbb3ded service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.321135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 818/1623] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:50.326244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c204269-679d-46f8-9730-0bdaab4cb41f None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.326244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c204269-679d-46f8-9730-0bdaab4cb41f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.327398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c204269-679d-46f8-9730-0bdaab4cb41f None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.328180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.328963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.329340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.330977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.337319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 819/1624] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 15:55:50 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:55:50.344673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c204269-679d-46f8-9730-0bdaab4cb41f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.352410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.354437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.359022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c204269-679d-46f8-9730-0bdaab4cb41f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.365322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.366186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.366573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.366914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.367933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.369757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 806/1625] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:50.370170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.370735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.383971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04251a28-3dc4-4f24-bedd-9f5b367493a0 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.390322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453a225-961c-47f2-80ec-012109906e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.391591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453a225-961c-47f2-80ec-012109906e76 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.391778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453a225-961c-47f2-80ec-012109906e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.392151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453a225-961c-47f2-80ec-012109906e76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.393015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453a225-961c-47f2-80ec-012109906e76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.395218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453a225-961c-47f2-80ec-012109906e76 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.405468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 820/1626] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/projects => generated 405 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:50.406945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.436768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 807/1627] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:50.436768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.436768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.436768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.458922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:50.460232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:50.475534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86861f86-a31d-4fd4-a0fb-120686431012 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.480176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:50.536905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ca802-e80f-4edc-a245-b3ab9c9821d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.537743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ca802-e80f-4edc-a245-b3ab9c9821d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.538181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ca802-e80f-4edc-a245-b3ab9c9821d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.538653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ca802-e80f-4edc-a245-b3ab9c9821d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.543295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 808/1628] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:50.544056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.609516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eafa89-4f97-4fe9-a82d-846942444609 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.618176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.618922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.619316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.619633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.621643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ca802-e80f-4edc-a245-b3ab9c9821d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.623925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ca802-e80f-4edc-a245-b3ab9c9821d5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.631576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:50.642053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 821/1629] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:50.643592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ec35e-f331-43d9-9912-1663cd2f5b59 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.644165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ec35e-f331-43d9-9912-1663cd2f5b59 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.644797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ec35e-f331-43d9-9912-1663cd2f5b59 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.646567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 822/1630] 10.222.0.17 () {54 vars in 962 bytes} [Wed Sep 7 15:55:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:55:50.647384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ec35e-f331-43d9-9912-1663cd2f5b59 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.668573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ec35e-f331-43d9-9912-1663cd2f5b59 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.674672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.675431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.675876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.676069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.676702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.679586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.686686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.688416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.699532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 809/1631] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:50.701049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.710838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 823/1632] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/users => generated 309 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:50.712002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.716549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c411ba3-29b1-4ce3-a74a-8f36fe19a975 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.722107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.723322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.723636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.723916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.724559 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.724881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072cdefc-aa0c-413e-b82b-2a664d786308 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.727720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.729442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da5c4a-1215-4095-9a1c-cd3d3f235242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.730965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5c4a-1215-4095-9a1c-cd3d3f235242 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.731391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5c4a-1215-4095-9a1c-cd3d3f235242 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.731792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da5c4a-1215-4095-9a1c-cd3d3f235242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.749735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5c4a-1215-4095-9a1c-cd3d3f235242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.751199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da5c4a-1215-4095-9a1c-cd3d3f235242 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.764763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 810/1633] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/users => generated 292 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:50.767952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 824/1634] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:50.767952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.767952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.767952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.770060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.787051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325544e-f6ad-4db5-9947-5808c69eff98 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.790273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:50.791670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:50.794985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.796006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.796380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.796766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.797761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.801868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.804103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 825/1635] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Sep 7 15:55:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:50.805245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.815741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 811/1636] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:50.816899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.820225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88656c9-6d72-4232-8685-5ddd64b96638 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.827375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.828194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.828485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.828814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.829750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.832040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.841062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fcc1d-4c6b-4d3a-acaa-749ee5254307 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.846084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 826/1637] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:50.847576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.854489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.855504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.856437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.856763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.859896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2650cb3c-1098-4f21-a52f-f12ac7b799db admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:50.866444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:50.867300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:50.867796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:50.868202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] PATH_INFO: `/v3/projects/0e681c43013343378cc98efc438b968c/users/fe951db85bcc4e24b9fcf58a3b901da1/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:50.885567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e681c43013343378cc98efc438b968c, user_id=fe951db85bcc4e24b9fcf58a3b901da1, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.888629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.914832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 827/1638] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:50 2022] PUT /identity/v3/projects/0e681c43013343378cc98efc438b968c/users/fe951db85bcc4e24b9fcf58a3b901da1/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:50.917157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:50.920921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:50.954990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:50.959229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:50.971112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 812/1639] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:50.971686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:51.002735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c22d831-2786-4365-bf5e-30fb6699ee82 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.096122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccace4c-0069-4212-a4b1-ac31b720ff45 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.107385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.108437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.108836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.109266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] PATH_INFO: `/v3/projects/248e24fd1f2549639337c182f244809f/users/d8a85d8812ab49c0bcff379b170bd2a6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.109389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.109910 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.110255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.110530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.123986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.130011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=248e24fd1f2549639337c182f244809f, user_id=d8a85d8812ab49c0bcff379b170bd2a6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.134338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.136220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.168558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 813/1640] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:51.180637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:51.185267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 828/1641] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:50 2022] PUT /identity/v3/projects/248e24fd1f2549639337c182f244809f/users/d8a85d8812ab49c0bcff379b170bd2a6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:51.189085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:51.215349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157972e-5c71-4a2c-8b30-99b37298137b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.338447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da62f01-3c21-4ac6-a198-0332e214bd4b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.349921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.350607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.350897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.351173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.352122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.358960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.384315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21715e24-c21c-4d6a-be56-8b3321dca6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.385159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21715e24-c21c-4d6a-be56-8b3321dca6f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.386001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 829/1642] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:55:51.386761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21715e24-c21c-4d6a-be56-8b3321dca6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.387187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21715e24-c21c-4d6a-be56-8b3321dca6f5 admin admin] PATH_INFO: `/v3/projects/0e681c43013343378cc98efc438b968c/users/fe951db85bcc4e24b9fcf58a3b901da1/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.388144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:51.393653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21715e24-c21c-4d6a-be56-8b3321dca6f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e681c43013343378cc98efc438b968c, user_id=fe951db85bcc4e24b9fcf58a3b901da1, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.398231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21715e24-c21c-4d6a-be56-8b3321dca6f5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.412022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c8b65-fa5e-4189-aa80-34ee47209ed5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.423829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a7b64-e900-4aba-b3df-95f10b173ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.424711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a7b64-e900-4aba-b3df-95f10b173ded admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.424997 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a7b64-e900-4aba-b3df-95f10b173ded admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.425271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a7b64-e900-4aba-b3df-95f10b173ded admin admin] PATH_INFO: `/v3/projects/248e24fd1f2549639337c182f244809f/users/d8a85d8812ab49c0bcff379b170bd2a6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.429536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 814/1643] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:51 2022] PUT /identity/v3/projects/0e681c43013343378cc98efc438b968c/users/fe951db85bcc4e24b9fcf58a3b901da1/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:51.432089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a7b64-e900-4aba-b3df-95f10b173ded admin admin] RBAC: Authorizing `identity:create_grant(project_id=248e24fd1f2549639337c182f244809f, user_id=d8a85d8812ab49c0bcff379b170bd2a6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.438867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a7b64-e900-4aba-b3df-95f10b173ded admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.445111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.445941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:51.446352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.446740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.465096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 830/1644] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:51 2022] PUT /identity/v3/projects/248e24fd1f2549639337c182f244809f/users/d8a85d8812ab49c0bcff379b170bd2a6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:51.476435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccae68-78d9-4df8-b22c-10f2967e5f1d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.477072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccae68-78d9-4df8-b22c-10f2967e5f1d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.477493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ccae68-78d9-4df8-b22c-10f2967e5f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.538376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-98ccae68-78d9-4df8-b22c-10f2967e5f1d None None] MFA Rules not processed for user `d8a85d8812ab49c0bcff379b170bd2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:51.539474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-98ccae68-78d9-4df8-b22c-10f2967e5f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:51.564689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] MFA Rules not processed for user `fe951db85bcc4e24b9fcf58a3b901da1`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:51.564689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:51.746985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 831/1645] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:51.748417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75211d-f4ce-44f5-bbd4-ce1daee58b2a None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.749178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75211d-f4ce-44f5-bbd4-ce1daee58b2a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.749463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75211d-f4ce-44f5-bbd4-ce1daee58b2a None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.758956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 832/1646] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 15:55:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:55:51.767286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.768161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.768439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.782441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:51.797435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 815/1647] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:51.798828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:51.810626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:51.811697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:51.875003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667daaef-73a6-4fc8-97fc-b3be9a65a491 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.881017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.882296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.882971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.883471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.900859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.902798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.908368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 833/1648] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:55:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:51.909357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:51.916889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 816/1649] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:51.920871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f818218-d350-40c1-93f0-8071754f9830 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:51.925337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:51.926660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:51.926995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:51.927270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:51.937631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:51.939169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:51.951073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 834/1650] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:51 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:52.079830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:52.096057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4796336-9ed8-4314-ad4d-0080df635fea service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:52.105312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:52.107064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:52.107351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:52.107687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:52.123398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:52.125483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:52.147702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 817/1651] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:52.377068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:52.398942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45876a87-8eb7-4943-9292-cf4cff4c011e service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:52.412586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:52.413356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:52.413760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:52.414040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:52.504743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:52.507579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:52.509230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:52.535518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 835/1652] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:52.580198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:52.642708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:52.700008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d63763-80fd-44eb-9d37-5290a37d54a5 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:52.712106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:52.714465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:52.714465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:52.714751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] PATH_INFO: `/v3/users/4feedcde86d04d2395b09ae65dee4ad9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:52.716869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4feedcde86d04d2395b09ae65dee4ad9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:52.719930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: WARNING py.warnings [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:55:52.719930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: warnings.warn(msg) Sep 07 15:55:52.719930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: Sep 07 15:55:52.722201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:52.789017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20fd45-a692-4234-9eab-221bfcde0bf1 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:52.802780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:52.803458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:52.803865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:52.804138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] PATH_INFO: `/v3/users/0d85459b723443ed8663854e03f3103d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:52.806325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d85459b723443ed8663854e03f3103d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:52.811986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:52.888631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4feedcde86d04d2395b09ae65dee4ad9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.903823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4feedcde86d04d2395b09ae65dee4ad9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.905046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4feedcde86d04d2395b09ae65dee4ad9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.918011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d85459b723443ed8663854e03f3103d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.932604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d85459b723443ed8663854e03f3103d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.958271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4feedcde86d04d2395b09ae65dee4ad9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.959585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0ef40ba6-ed12-4f62-a7e0-f894e9814ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4feedcde86d04d2395b09ae65dee4ad9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.999277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d85459b723443ed8663854e03f3103d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.999277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d85459b723443ed8663854e03f3103d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:52.999277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-d9a88116-34e2-46d0-a085-52c9d5949653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d85459b723443ed8663854e03f3103d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:53.024897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 818/1653] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:55:52 2022] DELETE /identity/v3/users/4feedcde86d04d2395b09ae65dee4ad9 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:53.029041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:53.029338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:53.029633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:53.042598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 836/1654] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:55:52 2022] DELETE /identity/v3/users/0d85459b723443ed8663854e03f3103d => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:53.048145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:53.048440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:53.048877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:53.059056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:53.086292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:53.087842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:53.093668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:53.095352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:53.223321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 819/1655] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:53.224040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 837/1656] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:53.251833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:53.251917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:53.251981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:53.252042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:53.302195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:53.402935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25afe9-c3cd-424c-8844-935d94e754c5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:53.408013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324d91f-6259-4154-9b51-30b9d705b474 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:53.412268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:53.412908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:53.413177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:53.413451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:53.414842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:53.417072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:53.417269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:53.418222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:53.445479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:53.446909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:53.447032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:53.450016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:53.475039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 820/1657] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:53.498244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 838/1658] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:54.327992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:54.329654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:54.350700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:54.352476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:54.373167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc235ca-fb2e-48b8-8436-58b7f6b02581 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:54.389129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d47316-cc59-42b7-85a4-cdf1fd2406c1 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:54.392092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:54.392743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:54.393024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:54.393299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] PATH_INFO: `/v3/projects/5d824a9763c149ff8e223edc7f13ed6a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:54.395289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d824a9763c149ff8e223edc7f13ed6a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:54.397266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:54.409928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:54.410633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:54.410918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:54.411195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] PATH_INFO: `/v3/projects/2d357053af334b5090309eae0d444d95` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:54.414554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d357053af334b5090309eae0d444d95)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:54.416134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: WARNING py.warnings [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 07 15:55:54.416134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: warnings.warn(msg) Sep 07 15:55:54.416134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: Sep 07 15:55:54.418253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:54.580694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d357053af334b5090309eae0d444d95', 'request_id': 'req-33128a99-33fc-4b97-b428-415682eb4736'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.605435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d357053af334b5090309eae0d444d95', 'request_id': 'req-33128a99-33fc-4b97-b428-415682eb4736'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.642039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d824a9763c149ff8e223edc7f13ed6a', 'request_id': 'req-e614702f-b53f-47b2-9edb-914baf799248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.655081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d357053af334b5090309eae0d444d95', 'request_id': 'req-33128a99-33fc-4b97-b428-415682eb4736'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.659508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d824a9763c149ff8e223edc7f13ed6a', 'request_id': 'req-e614702f-b53f-47b2-9edb-914baf799248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.681773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d824a9763c149ff8e223edc7f13ed6a', 'request_id': 'req-e614702f-b53f-47b2-9edb-914baf799248'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.699297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] The token cache is being invalidate because project 5d824a9763c149ff8e223edc7f13ed6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:55:54.699833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.701857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.702321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] The token cache is being invalidate because project 2d357053af334b5090309eae0d444d95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:55:54.702625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.704544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 821/1659] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:55:54 2022] DELETE /identity/v3/projects/5d824a9763c149ff8e223edc7f13ed6a => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:54.712141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:54.712269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-33128a99-33fc-4b97-b428-415682eb4736 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.715847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:54.719743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 839/1660] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:55:54 2022] DELETE /identity/v3/projects/2d357053af334b5090309eae0d444d95 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:54.729252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:54.729545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:54.729822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:54.774076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:54.819346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:54.825696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:54.858681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e614702f-b53f-47b2-9edb-914baf799248 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:54.865522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:54.866513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:54.867028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:54.869089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] PATH_INFO: `/v3/users/fe951db85bcc4e24b9fcf58a3b901da1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:54.870809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe951db85bcc4e24b9fcf58a3b901da1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:54.873130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:54.962182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe951db85bcc4e24b9fcf58a3b901da1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.976196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe951db85bcc4e24b9fcf58a3b901da1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:54.978014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe951db85bcc4e24b9fcf58a3b901da1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.004828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe951db85bcc4e24b9fcf58a3b901da1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.005935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3e2f8920-4a45-4fef-b5a8-a06534f1f93d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe951db85bcc4e24b9fcf58a3b901da1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.006998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 840/1661] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:55.008294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.011091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:55.027916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 822/1662] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:55:54 2022] DELETE /identity/v3/users/fe951db85bcc4e24b9fcf58a3b901da1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:55.030051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.030615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.031213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.050586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa76ae-e510-4c9c-81e4-def02e35b4b8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:55.064279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:55.066921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:55.175737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 823/1663] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:55.179807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.217529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4048d-65cf-4150-a7e0-85b27e2818a9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:55.225832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:55.249743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:55.260799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.261063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.261476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.262845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:55.263160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:55.266022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:55.275524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:55.276557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.277053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.277533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] PATH_INFO: `/v3/users/77b0a15642854df3a8131900cd0b1ee2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.280519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=77b0a15642854df3a8131900cd0b1ee2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:55.282970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:55.301800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 824/1664] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:55:55.310673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.340393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b62a37-7b4d-43c7-b871-e246cf3b27ef admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:55.358810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:55.361506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.361958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.363316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.365550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:55.373554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:55.467595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 825/1665] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:55 2022] POST /identity/v3/projects => generated 355 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:55.468884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77b0a15642854df3a8131900cd0b1ee2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.470365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.492711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77b0a15642854df3a8131900cd0b1ee2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.521588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:55.523916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77b0a15642854df3a8131900cd0b1ee2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.526382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77b0a15642854df3a8131900cd0b1ee2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.529363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-65248778-559c-4ed5-8462-4d7931590515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77b0a15642854df3a8131900cd0b1ee2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:55.564510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 841/1666] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:55:55 2022] DELETE /identity/v3/users/77b0a15642854df3a8131900cd0b1ee2 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:55.569684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.569977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.570371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.614988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:55.616531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:55.636100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade8703-8fa1-470a-998e-d9c49010008e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:55.758394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 842/1667] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:55.815263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:55.815929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.816210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.816526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.817174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:55.819189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:55.866381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 826/1668] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:55 2022] POST /identity/v3/users => generated 278 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:55.867284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.868984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:55.882957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.884782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:55.887708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e1840-7cf9-4c2b-af11-82683669a892 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:55.908301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:55.908947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.909246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.909964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.910204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:55.911873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:55.939992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 843/1669] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:55.944380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:55.965207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f48613-9a91-44d7-9b2d-e8ebae35365d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:55.981137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:55.981920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:55.982329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:55.982709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:55.983470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:55.985230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.005951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 844/1670] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:55:56.018240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:56.025952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8fd89d-a1ad-4414-88d4-76869734d5f0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.034201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14019884-bc1b-4ff5-937b-fc0556c5e04f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.034761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019884-bc1b-4ff5-937b-fc0556c5e04f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.035089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019884-bc1b-4ff5-937b-fc0556c5e04f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.035362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14019884-bc1b-4ff5-937b-fc0556c5e04f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.058477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c5491-db72-4753-9817-97b024bea41a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.061002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14019884-bc1b-4ff5-937b-fc0556c5e04f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:56.062496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14019884-bc1b-4ff5-937b-fc0556c5e04f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.080680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.085465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 827/1671] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:55:56.095891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.096526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.096925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] PATH_INFO: `/v3/projects/eb601b43b8cf4480839d87b7eb24d1b7/users/431d40a24b3e4ecdb0b5154543ec515f/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.124636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb601b43b8cf4480839d87b7eb24d1b7, user_id=431d40a24b3e4ecdb0b5154543ec515f, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:56.128098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.159896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 845/1672] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:56 2022] PUT /identity/v3/projects/eb601b43b8cf4480839d87b7eb24d1b7/users/431d40a24b3e4ecdb0b5154543ec515f/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:56.170520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.170848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.171083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.220822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] MFA Rules not processed for user `431d40a24b3e4ecdb0b5154543ec515f`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:56.223227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:56.313685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:56.345243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 828/1673] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:56 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:56.349593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:56.419819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:56.537749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbff2d-012b-4df6-ae64-904dba4d2bf1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.555909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.557783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.558229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.558796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.560287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:56.568645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.690125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:56.694654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 846/1674] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:56 2022] POST /identity/v3/projects => generated 355 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:56.712141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36488f3c-bce6-46bc-8819-d8a29b2e061a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.818883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.819505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.819813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.820084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.820779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:56.822899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.862620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:56.864510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:56.881627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 829/1675] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:56 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:56.885310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:56.899421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b7a8d0-250c-496c-9027-5b76f6a142a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.912352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.913114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.913297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.913641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] PATH_INFO: `/v3/projects/0e681c43013343378cc98efc438b968c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.914740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e681c43013343378cc98efc438b968c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:56.916537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2044d-d281-477d-818d-f589f218cfad admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.916748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.922556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.923427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.924007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.924626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:56.926322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:56.928560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:56.941094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 830/1676] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:56.949102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:56.969445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c8966a-0fc5-4c7b-b184-c317f9cc2538 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:56.978477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:56.979308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:56.979711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:56.980041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] PATH_INFO: `/v3/projects/0def520da8144582a9db2b3dfde22896/users/91ea2ffacbaa4c43b7c5cf6c6cfb1924/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.018022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0def520da8144582a9db2b3dfde22896, user_id=91ea2ffacbaa4c43b7c5cf6c6cfb1924, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:57.021900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:57.024135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e681c43013343378cc98efc438b968c', 'request_id': 'req-45966810-c8b0-49aa-8a6d-615c9874d4ad'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:57.043930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e681c43013343378cc98efc438b968c', 'request_id': 'req-45966810-c8b0-49aa-8a6d-615c9874d4ad'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:57.051581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 831/1677] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:56 2022] PUT /identity/v3/projects/0def520da8144582a9db2b3dfde22896/users/91ea2ffacbaa4c43b7c5cf6c6cfb1924/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:57.052677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:57.067891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e681c43013343378cc98efc438b968c', 'request_id': 'req-45966810-c8b0-49aa-8a6d-615c9874d4ad'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:57.083819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] The token cache is being invalidate because project 0e681c43013343378cc98efc438b968c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:55:57.084247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:57.085111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-45966810-c8b0-49aa-8a6d-615c9874d4ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:57.087055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 847/1678] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:55:56 2022] DELETE /identity/v3/projects/0e681c43013343378cc98efc438b968c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:55:57.092898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2787-3d31-40a0-86b0-84ea857303f1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.093348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2787-3d31-40a0-86b0-84ea857303f1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.093877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b2787-3d31-40a0-86b0-84ea857303f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.119294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4f4b2787-3d31-40a0-86b0-84ea857303f1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:57.120734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4f4b2787-3d31-40a0-86b0-84ea857303f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:57.214011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeabb862-6303-4ebf-a135-1c56a74d3603 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:57.229286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:57.229915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.237381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.238162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.241129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:57.251350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:57.279015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 848/1679] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:57.284453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.285536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.285956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.289431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 832/1680] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:55:57.292574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:57.294744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:57.331131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5edd5a-7caa-4499-a16e-43f2ea9802fa admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:57.336046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:57.337846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:57.345064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717cf6cc-9ae9-4350-a735-2de70ca5c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:57.346006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cf6cc-9ae9-4350-a735-2de70ca5c692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.347355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cf6cc-9ae9-4350-a735-2de70ca5c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.348363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717cf6cc-9ae9-4350-a735-2de70ca5c692 admin admin] PATH_INFO: `/v3/projects/0def520da8144582a9db2b3dfde22896/users/91ea2ffacbaa4c43b7c5cf6c6cfb1924/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.351312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cf6cc-9ae9-4350-a735-2de70ca5c692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0def520da8144582a9db2b3dfde22896, user_id=91ea2ffacbaa4c43b7c5cf6c6cfb1924, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:57.356788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717cf6cc-9ae9-4350-a735-2de70ca5c692 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:57.361750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 849/1681] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:57.365572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:57.380597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1d8cf-8aaa-4112-8f4e-fb3192537fd6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:57.382678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 833/1682] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:57 2022] PUT /identity/v3/projects/0def520da8144582a9db2b3dfde22896/users/91ea2ffacbaa4c43b7c5cf6c6cfb1924/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:57.392966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.394069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.395028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.420012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:57.437070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] MFA Rules not processed for user `91ea2ffacbaa4c43b7c5cf6c6cfb1924`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:57.439443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:57.451880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:57.558736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:57.560372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.560788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.561490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.564854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:55:57.566672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:57.570622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:57.608586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 850/1683] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:55:57.611874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:57.626224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 834/1684] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:57 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:55:57.634109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53277a3a-8437-41a4-a68e-9c89ef3cc6a6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:57.639606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:57.641664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:57.651839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:57.652551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.653347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.654856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.655674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:57.665334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:57.735317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f290986a-4f31-4339-a2b8-86eab30fd07c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:57.735709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 851/1685] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:57 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:57.739860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:57.903816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:57.906170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.906298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.906728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.936165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1151de4d-b0b6-4e9a-9c33-ec684b949f35 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:57.955822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:57.956626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:57.957293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:57.957293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:57.957888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:57.968064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.039457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 852/1686] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:57 2022] POST /identity/v3/users => generated 290 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:58.043534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.066835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed0ec82-6a7f-44d9-8624-46fa6f12ace4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.075311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.076299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.076996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.078110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.079134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.081011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.097813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 853/1687] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:58.108009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.121629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df703f4-a092-47b2-9b2c-583fad403bcd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.127199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.128482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.129035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.129408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.130157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.131953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.137828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.139307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.148226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 854/1688] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:58.150408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.152738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:58.169264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 835/1689] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:57 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:58.170123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.181969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c860d8d3-989b-41fd-933c-d17ec877cbe3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.189156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.190773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.191430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.191828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] PATH_INFO: `/v3/projects/755d15a7feea47eea8bfaaa4da4c4a2e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.193136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=755d15a7feea47eea8bfaaa4da4c4a2e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.195387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.199503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e453b3b2-b2d5-4475-87c4-5e02d8f459e5 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.213263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffbaff-5f28-4b6d-a446-e7fc6773044d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.213693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffbaff-5f28-4b6d-a446-e7fc6773044d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.213870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffbaff-5f28-4b6d-a446-e7fc6773044d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.214235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffbaff-5f28-4b6d-a446-e7fc6773044d admin admin] PATH_INFO: `/v3/projects/7f5d790baaf8420a811db6bf113a9a82/users/f0fb165c490046e8b6044c329b3404fc/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.245629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:58.254345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffbaff-5f28-4b6d-a446-e7fc6773044d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5d790baaf8420a811db6bf113a9a82, user_id=f0fb165c490046e8b6044c329b3404fc, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.268408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffbaff-5f28-4b6d-a446-e7fc6773044d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.298243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 836/1690] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:58 2022] PUT /identity/v3/projects/7f5d790baaf8420a811db6bf113a9a82/users/f0fb165c490046e8b6044c329b3404fc/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:58.299667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755d15a7feea47eea8bfaaa4da4c4a2e', 'request_id': 'req-5730af39-059a-4098-8aa4-8a1222439e3e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.306432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:58.307709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.308003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.308577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.319279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755d15a7feea47eea8bfaaa4da4c4a2e', 'request_id': 'req-5730af39-059a-4098-8aa4-8a1222439e3e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.330678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755d15a7feea47eea8bfaaa4da4c4a2e', 'request_id': 'req-5730af39-059a-4098-8aa4-8a1222439e3e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.374455 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] The token cache is being invalidate because project 755d15a7feea47eea8bfaaa4da4c4a2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:55:58.379357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.380635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.384621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 855/1691] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:55:58 2022] DELETE /identity/v3/projects/755d15a7feea47eea8bfaaa4da4c4a2e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:58.386771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.389508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:58.390848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] MFA Rules not processed for user `f0fb165c490046e8b6044c329b3404fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:58.397640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:58.556063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730af39-059a-4098-8aa4-8a1222439e3e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.576776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 837/1692] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:58.576776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.578256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:58.626046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.626684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.626962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.627258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.629585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:58.712194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.713870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.730536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 856/1693] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:58 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:55:58.731501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.732692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491edccf-97da-4303-a0d1-df4456ddc07e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.733534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:58.749530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.750043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e16d-b0fe-47e6-98f0-30d1a01e4a30 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:58.750220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.750474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.750748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] PATH_INFO: `/v3/users/ec54877e04804aedafc3a49cef81511d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.752572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec54877e04804aedafc3a49cef81511d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.754012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.760431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:58.761261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:58.761713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:58.762081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:58.762848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:58.768993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:58.847875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 857/1694] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:55:58 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:58.849749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:58.859968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec54877e04804aedafc3a49cef81511d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.882168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec54877e04804aedafc3a49cef81511d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.882860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec54877e04804aedafc3a49cef81511d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.909867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec54877e04804aedafc3a49cef81511d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.911182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec54877e04804aedafc3a49cef81511d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:55:58.930607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:58.943254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 838/1695] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:55:58 2022] DELETE /identity/v3/users/ec54877e04804aedafc3a49cef81511d => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:58.947245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:55:59.074349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918016d-0aa7-4310-866a-7fe90f8305de admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:59.174145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:59.181788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:59.184027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:59.184475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:59.186746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:59.196976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:59.257495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 858/1696] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:55:58 2022] POST /identity/v3/users => generated 297 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:55:59.261528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:59.263320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:59.279178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe2ea3d-0d63-4c59-8bf4-809e228c351a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:59.287504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:59.289712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:59.290866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:59.291784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:59.295156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:59.297259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:59.311179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 839/1697] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:59.317919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:59.335049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7388d6a1-bc77-47bb-ba12-21c9ab3a991e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:59.353433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:59.355023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:59.358667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:59.359195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] PATH_INFO: `/v3/projects/5952842d022643c9b8abfbf36fd0a205/users/e160d012676e4a32a8cde98fb3db1567/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:59.395076 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5952842d022643c9b8abfbf36fd0a205, user_id=e160d012676e4a32a8cde98fb3db1567, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:59.398231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:59.430705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 859/1698] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:59 2022] PUT /identity/v3/projects/5952842d022643c9b8abfbf36fd0a205/users/e160d012676e4a32a8cde98fb3db1567/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:55:59.435743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:59.453465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d2fbc6-4039-46b4-b3ed-539e10b45c17 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:59.574013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1bec7-0f37-42d1-84b4-73c33d9b9ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:59.574806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1bec7-0f37-42d1-84b4-73c33d9b9ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:59.575193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1bec7-0f37-42d1-84b4-73c33d9b9ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:59.575654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1bec7-0f37-42d1-84b4-73c33d9b9ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:59.576411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1bec7-0f37-42d1-84b4-73c33d9b9ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:59.579156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1bec7-0f37-42d1-84b4-73c33d9b9ac5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:59.603923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:59.604237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 840/1699] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:55:59.631522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97974d9d-9fc9-49e2-8735-ccfcf4ba88a7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:55:59.654364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:55:59.655875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:59.656551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:59.656831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] PATH_INFO: `/v3/projects/5952842d022643c9b8abfbf36fd0a205/users/e160d012676e4a32a8cde98fb3db1567/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:59.660580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5952842d022643c9b8abfbf36fd0a205, user_id=e160d012676e4a32a8cde98fb3db1567, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:55:59.667615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:55:59.714364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 860/1700] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:55:59 2022] PUT /identity/v3/projects/5952842d022643c9b8abfbf36fd0a205/users/e160d012676e4a32a8cde98fb3db1567/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:55:59.734269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:55:59.734541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:55:59.734919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:55:59.788591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] MFA Rules not processed for user `e160d012676e4a32a8cde98fb3db1567`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:55:59.790708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:55:59.927752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 841/1701] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:55:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:55:59.934215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:55:59.937043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:00.050323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca317679-6b5d-449d-bbb0-618890ab80c3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:00.079097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:00.079957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.080496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.080754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.097011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:00.100396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:00.112399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:00.115316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:00.141676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 861/1702] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:55:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:00.259865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf60b0-eaab-4eef-a95a-e8f931a667ad None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:00.266166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:00.267050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.267501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.267892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] PATH_INFO: `/v3/projects/0fbad6f2a321449395f652e6f8fe9bd0` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.270271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fbad6f2a321449395f652e6f8fe9bd0)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:00.272529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:00.295866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:00.298172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:00.300403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:00.338663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dcec8f-0265-4941-a858-c342310d72a2 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:00.349598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:00.350413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.351086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.351365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.355200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:00.415481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fbad6f2a321449395f652e6f8fe9bd0', 'request_id': 'req-a67a2809-4ecb-4958-a3a3-be9ea4112381'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:00.424146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:00.431440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fbad6f2a321449395f652e6f8fe9bd0', 'request_id': 'req-a67a2809-4ecb-4958-a3a3-be9ea4112381'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:00.446292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fbad6f2a321449395f652e6f8fe9bd0', 'request_id': 'req-a67a2809-4ecb-4958-a3a3-be9ea4112381'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:00.468926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] The token cache is being invalidate because project 0fbad6f2a321449395f652e6f8fe9bd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:00.469210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:00.471172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a67a2809-4ecb-4958-a3a3-be9ea4112381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:00.473661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 842/1703] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:00 2022] DELETE /identity/v3/projects/0fbad6f2a321449395f652e6f8fe9bd0 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:00.479420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.479420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.480719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.508541 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:00.510040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dec120-c051-4af4-b39a-ed6910b54646 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:00.528665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:00.529937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:00.533291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 862/1704] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:00.675338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 843/1705] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:00.684256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.684837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.684837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.787421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:00.794461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:00.826764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 863/1706] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:00.827896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:00.844541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c5aa8-28d9-4e1b-bf9b-f1a39d741ab6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:00.866666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:00.872232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.873177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.874980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.876122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:00.878067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:00.884714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:00.929937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 844/1707] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:00.934786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:00.952824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54d5a2-c961-467a-97d8-932db40a4bda None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:00.966594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:00.967743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:00.968221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:00.968631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:00.970650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:00.975895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:01.047840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 864/1708] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:00 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:01.049946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:01.067025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986ab93-5a79-4463-8a78-4b3acd7598de admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:01.288259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:01.288901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:01.289299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:01.289639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:01.290757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:01.296981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:01.377763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 845/1709] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:01 2022] POST /identity/v3/users => generated 290 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:01.379153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:01.402414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ea064-b1fc-4bc3-99db-79aa4e56df42 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:01.428018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:01.431679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:01.431977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:01.432259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:01.432941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:01.448253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:01.512362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:01.513086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 865/1710] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:01.533070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbf678-20cf-4583-88f2-9a1869b07e94 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:01.552557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce6047a-ba23-4aff-b859-9f4c59d0f209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:01.553133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6047a-ba23-4aff-b859-9f4c59d0f209 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:01.553417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6047a-ba23-4aff-b859-9f4c59d0f209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:01.553693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce6047a-ba23-4aff-b859-9f4c59d0f209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:01.554359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6047a-ba23-4aff-b859-9f4c59d0f209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:01.562263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce6047a-ba23-4aff-b859-9f4c59d0f209 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:01.662362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 846/1711] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:01.665801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:01.685664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302c1987-6c1b-4423-b29f-115c3cb9b56d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:01.704334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:01.705033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:01.705352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:01.705632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] PATH_INFO: `/v3/projects/993a1f1d86a741dda11fdd32c27091b2/users/9e2169ca670b423ca78289bfbef29e8e/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:01.758341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993a1f1d86a741dda11fdd32c27091b2, user_id=9e2169ca670b423ca78289bfbef29e8e, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:01.762797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:01.794385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 866/1712] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:01 2022] PUT /identity/v3/projects/993a1f1d86a741dda11fdd32c27091b2/users/9e2169ca670b423ca78289bfbef29e8e/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:01.805599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:01.805777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:01.805993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:01.817692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:01.887346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] MFA Rules not processed for user `9e2169ca670b423ca78289bfbef29e8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:01.889589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:02.054833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 847/1713] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:02.059495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:02.201078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d18dbb-3397-49ee-a2db-e0cb2dfdcdc7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:02.211304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:02.213067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:02.213471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:02.214176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:02.217898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:02.224988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:02.300324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 867/1714] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:02 2022] POST /identity/v3/projects => generated 383 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:02.303918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:02.321430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a273f0-8096-4e0b-bf5e-fb1af6ac995e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:02.513244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:02.513892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:02.514175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:02.514453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:02.515114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:02.517234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:02.580672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 848/1715] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:02 2022] POST /identity/v3/users => generated 298 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:02.583866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:02.613624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5244ea-5158-43df-8dc9-f811465864cf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:02.628194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:02.629684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:02.631934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:02.632348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:02.634824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:02.640835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:02.698817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:02.709707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 868/1716] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:02.713461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712aafc9-07eb-458b-9657-23d0c8519b85 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:02.737287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe36034-da31-4837-8882-77936d3244a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:02.738809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36034-da31-4837-8882-77936d3244a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:02.739500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36034-da31-4837-8882-77936d3244a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:02.740242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe36034-da31-4837-8882-77936d3244a1 admin admin] PATH_INFO: `/v3/projects/aeee51439ef24c45b2191fc62bbee14a/users/c441f51e5acf46d5bd95dab9f3f2d177/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:02.770876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36034-da31-4837-8882-77936d3244a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeee51439ef24c45b2191fc62bbee14a, user_id=c441f51e5acf46d5bd95dab9f3f2d177, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:02.775061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe36034-da31-4837-8882-77936d3244a1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:02.820715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 849/1717] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:02 2022] PUT /identity/v3/projects/aeee51439ef24c45b2191fc62bbee14a/users/c441f51e5acf46d5bd95dab9f3f2d177/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:02.824904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:02.841038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba81a5-ebd1-4450-94d2-09ffee724cbb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:02.987277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:03.045752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e2fe5-39a5-430f-b498-7f3af0d24ff2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:03.046064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e2fe5-39a5-430f-b498-7f3af0d24ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:03.046483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e2fe5-39a5-430f-b498-7f3af0d24ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:03.113233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f48e2fe5-39a5-430f-b498-7f3af0d24ff2 None None] MFA Rules not processed for user `de43a221bd8148b39eabc978b97f0e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:03.114838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f48e2fe5-39a5-430f-b498-7f3af0d24ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:03.122372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:03.123006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:03.123288 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:03.123568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:03.133952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:03.135678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:03.155436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 869/1718] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:03.157780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:03.181631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61218126-3fa8-40d2-84ce-d44e922114ce admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:03.194247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38252471-a934-47eb-a3fe-92ec05b28788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:03.194858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252471-a934-47eb-a3fe-92ec05b28788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:03.195137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252471-a934-47eb-a3fe-92ec05b28788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:03.195408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38252471-a934-47eb-a3fe-92ec05b28788 admin admin] PATH_INFO: `/v3/projects/aeee51439ef24c45b2191fc62bbee14a/users/c441f51e5acf46d5bd95dab9f3f2d177/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:03.201365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38252471-a934-47eb-a3fe-92ec05b28788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeee51439ef24c45b2191fc62bbee14a, user_id=c441f51e5acf46d5bd95dab9f3f2d177, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:03.208149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38252471-a934-47eb-a3fe-92ec05b28788 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:03.252965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 870/1719] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:03 2022] PUT /identity/v3/projects/aeee51439ef24c45b2191fc62bbee14a/users/c441f51e5acf46d5bd95dab9f3f2d177/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:03.258996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:03.259170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:03.259753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:03.259899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:03.300171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 850/1720] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:03.312296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:03.312634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:03.312918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:03.384746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] MFA Rules not processed for user `de43a221bd8148b39eabc978b97f0e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:03.387214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:03.409099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] MFA Rules not processed for user `c441f51e5acf46d5bd95dab9f3f2d177`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:03.411729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:03.699065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 871/1721] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 442 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:03.699688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 851/1722] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:03.707822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:03.719210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:03.814007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:03.815709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:03.965937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d096033b-5a8c-4852-8d95-4b56894e807c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:04.018649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:04.021568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:04.022007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:04.022560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:04.030205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b9a1c-0079-44b1-85ef-021562fd089f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:04.038911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:04.041083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:04.049124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d2068-f16e-42a3-863c-6e4407803ca1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:04.049820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d2068-f16e-42a3-863c-6e4407803ca1 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:04.050130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d2068-f16e-42a3-863c-6e4407803ca1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:04.057337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d2068-f16e-42a3-863c-6e4407803ca1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:04.078473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 872/1723] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:04.088227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d2068-f16e-42a3-863c-6e4407803ca1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:04.093286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34d2068-f16e-42a3-863c-6e4407803ca1 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:04.140464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 852/1724] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:04.477914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:04.480382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:04.501437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316ed2a-4bcf-450e-87fd-1f36e7924250 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:04.516554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f618637b-c0b2-4d87-b65a-26e8ac5486b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:04.517558 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618637b-c0b2-4d87-b65a-26e8ac5486b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:04.517961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618637b-c0b2-4d87-b65a-26e8ac5486b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:04.518487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f618637b-c0b2-4d87-b65a-26e8ac5486b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:04.579360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:04.604013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:04.604301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:04.604580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:04.636346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618637b-c0b2-4d87-b65a-26e8ac5486b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:04.637681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f618637b-c0b2-4d87-b65a-26e8ac5486b1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:04.676439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 873/1725] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:04 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:04.687207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] MFA Rules not processed for user `d7e96593e2814e8b9f7c0f320aa17c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:04.688708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:04.834785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a128-2017-44f3-9bc2-7d2f5cb7e8f7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:04.835678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a128-2017-44f3-9bc2-7d2f5cb7e8f7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:04.836091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e2a128-2017-44f3-9bc2-7d2f5cb7e8f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:04.862209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 853/1726] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:04.879651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:04.881152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:04.892344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-29e2a128-2017-44f3-9bc2-7d2f5cb7e8f7 None None] MFA Rules not processed for user `7dc84ece6eb84315b54defd89be8a95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:04.894835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-29e2a128-2017-44f3-9bc2-7d2f5cb7e8f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:04.910378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418ccf1-e882-4ac1-b940-6c9ca9e05693 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:04.924623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95167279-1280-45ae-84f7-39eb98ce82ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:04.925385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95167279-1280-45ae-84f7-39eb98ce82ab service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:04.925979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95167279-1280-45ae-84f7-39eb98ce82ab service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:04.926383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95167279-1280-45ae-84f7-39eb98ce82ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:04.945248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95167279-1280-45ae-84f7-39eb98ce82ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:04.947066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95167279-1280-45ae-84f7-39eb98ce82ab service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:04.966199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 854/1727] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:05.010436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e817-5634-437c-abab-05871a42ca03 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:05.010840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e817-5634-437c-abab-05871a42ca03 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:05.011242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277e817-5634-437c-abab-05871a42ca03 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:05.012748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 855/1728] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 15:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:56:05.020310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:05.020310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:05.020796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:05.039946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 874/1729] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:04 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:05.048229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:05.049768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:05.050059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:05.060277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:05.081779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:05.083713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:05.101050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] MFA Rules not processed for user `7dc84ece6eb84315b54defd89be8a95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:05.102664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:05.125929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 875/1730] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:05.189702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:05.227685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eac50-8879-4a86-b38f-e14a4cf17a13 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:05.263392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 856/1731] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:56:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:05.265831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ab56f-90bd-4dbb-87da-47df089c563d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:05.266513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ab56f-90bd-4dbb-87da-47df089c563d service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:05.266804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ab56f-90bd-4dbb-87da-47df089c563d service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:05.267249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ab56f-90bd-4dbb-87da-47df089c563d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:05.287382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:05.289454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:05.291857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ab56f-90bd-4dbb-87da-47df089c563d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:05.293865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ab56f-90bd-4dbb-87da-47df089c563d service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:05.305471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 876/1732] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:05.313879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693e393-5d14-4529-b5b2-fb3fcacee3f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:05.326496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7d6de-cb65-4714-a080-be8d6ea56db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:05.328571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7d6de-cb65-4714-a080-be8d6ea56db3 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:05.330205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7d6de-cb65-4714-a080-be8d6ea56db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:05.330993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7d6de-cb65-4714-a080-be8d6ea56db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:05.350516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7d6de-cb65-4714-a080-be8d6ea56db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:05.353442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7d6de-cb65-4714-a080-be8d6ea56db3 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:05.365459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 857/1733] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:06.007803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b45df-3ad1-4981-b9b2-cad93e42ad06 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:06.008255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b45df-3ad1-4981-b9b2-cad93e42ad06 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:06.008551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b45df-3ad1-4981-b9b2-cad93e42ad06 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:06.010331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 877/1734] 10.222.0.17 () {54 vars in 912 bytes} [Wed Sep 7 15:56:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:56:06.023902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:06.025597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:06.026611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:06.078407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:06.080808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:06.120854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 858/1735] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:56:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:06.133903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b45df-3ad1-4981-b9b2-cad93e42ad06 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:06.135707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-831b45df-3ad1-4981-b9b2-cad93e42ad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:06.163533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b45df-3ad1-4981-b9b2-cad93e42ad06 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:06.189202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4a7e24-12cb-4969-8fbf-0c9190ec0473 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:06.190324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a7e24-12cb-4969-8fbf-0c9190ec0473 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:06.191537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a7e24-12cb-4969-8fbf-0c9190ec0473 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:06.192914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4a7e24-12cb-4969-8fbf-0c9190ec0473 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:06.209231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a7e24-12cb-4969-8fbf-0c9190ec0473 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:06.211078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4a7e24-12cb-4969-8fbf-0c9190ec0473 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:06.222596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 878/1736] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:06.262020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:06.263615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:06.413248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eeb1fd-58cf-4e70-8d23-b49557810a24 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:06.420289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:06.420922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:06.421210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:06.421485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:06.423287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:06.447356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:06.449119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f15ff0-185a-4aa8-98df-a18e47edf745 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:06.461078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 859/1737] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:07.275733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ef02a-6257-4efe-aa28-3aae1926575d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:07.276256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ef02a-6257-4efe-aa28-3aae1926575d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:07.277138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78ef02a-6257-4efe-aa28-3aae1926575d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:07.312218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e78ef02a-6257-4efe-aa28-3aae1926575d None None] MFA Rules not processed for user `7dc84ece6eb84315b54defd89be8a95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:07.313644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e78ef02a-6257-4efe-aa28-3aae1926575d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:07.323143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 879/1738] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:07 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:09.574199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:09.575882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:09.576678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:09.608067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] MFA Rules not processed for user `d7e96593e2814e8b9f7c0f320aa17c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:09.610180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:09.629125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 860/1739] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:13.539721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6442-570f-4af7-9339-4b8f149c27f6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:13.540226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6442-570f-4af7-9339-4b8f149c27f6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:13.540323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6d6442-570f-4af7-9339-4b8f149c27f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:13.724308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5e6d6442-570f-4af7-9339-4b8f149c27f6 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:13.728597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5e6d6442-570f-4af7-9339-4b8f149c27f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:13.895933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:13.920833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 880/1740] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:56:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:14.119398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:14.124665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:14.301867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:14.319802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea33f8a-54c6-42db-80fd-db0e06d5e656 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:14.325846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:14.326507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:14.326939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:14.327351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:14.338413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:14.340862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:14.350179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 861/1741] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:56:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:56:15.017776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81361b-98bd-4d03-bd36-9793e8e4c5cc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:15.018404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81361b-98bd-4d03-bd36-9793e8e4c5cc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:15.024102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81361b-98bd-4d03-bd36-9793e8e4c5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:15.134249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4f81361b-98bd-4d03-bd36-9793e8e4c5cc None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:15.157661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4f81361b-98bd-4d03-bd36-9793e8e4c5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:15.193819 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 881/1742] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Sep 7 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:15.293723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:15.294920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:15.412750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c683d9b8-a7f4-4f70-a7b0-6ef8f8c769f3 service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:15.419486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:15.420434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:15.421038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:15.421485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] PATH_INFO: `/v3/users/53642660b6584d5e89f8e3e9c0d3a301` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:15.422991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=53642660b6584d5e89f8e3e9c0d3a301)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:15.425429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:15.505275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53642660b6584d5e89f8e3e9c0d3a301'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:15.526409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53642660b6584d5e89f8e3e9c0d3a301'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:15.533061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53642660b6584d5e89f8e3e9c0d3a301'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:15.596209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53642660b6584d5e89f8e3e9c0d3a301'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:15.600793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53642660b6584d5e89f8e3e9c0d3a301'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:15.657468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 862/1743] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:15 2022] DELETE /identity/v3/users/53642660b6584d5e89f8e3e9c0d3a301 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:15.659228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:15.660978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:15.661442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:15.694389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:15.696830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:15.757550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:15.759301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:15.840147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b847c-7fe9-4b28-be45-789609d8d9eb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:15.846192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:15.847401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:15.847990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:15.848550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:15.853715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:15.868128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:15.871868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:15.887387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 863/1744] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:15.983024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 882/1745] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:16.018056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:16.023617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:16.031968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:16.034126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:16.055847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:16.075671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66528336-3659-40fc-89df-c6afca8dd422 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:16.090943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be4773-6a16-40c5-8bbb-8c977040d00a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:16.095775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:16.096589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:16.099047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:16.099454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] PATH_INFO: `/v3/users/9e2169ca670b423ca78289bfbef29e8e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:16.102895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e2169ca670b423ca78289bfbef29e8e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:16.106923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:16.135677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:16.136330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:16.136617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:16.136895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:16.195317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:16.199354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:16.285133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 883/1746] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:16.292722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:16.337633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85784863-a724-48a8-9ea1-26596fbc527c service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:16.368390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3120bb-8d99-40d8-b9ee-88deece083c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:16.369335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3120bb-8d99-40d8-b9ee-88deece083c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:16.369779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3120bb-8d99-40d8-b9ee-88deece083c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:16.370211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3120bb-8d99-40d8-b9ee-88deece083c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:16.372082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e2169ca670b423ca78289bfbef29e8e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:16.397270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e2169ca670b423ca78289bfbef29e8e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:16.405296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e2169ca670b423ca78289bfbef29e8e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:16.410928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3120bb-8d99-40d8-b9ee-88deece083c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:16.412617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3120bb-8d99-40d8-b9ee-88deece083c5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:16.450402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e2169ca670b423ca78289bfbef29e8e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:16.451857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e2169ca670b423ca78289bfbef29e8e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:16.465845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 884/1747] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:16.483600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 864/1748] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:16 2022] DELETE /identity/v3/users/9e2169ca670b423ca78289bfbef29e8e => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:16.488947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:16.489723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:16.490088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:16.521782 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:16.539674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:16.699852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 885/1749] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:16.816461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:16.820688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:17.009107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ba771a-ac77-4ccf-a95e-7afb8d336f29 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:17.017082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:17.019406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:17.019906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:17.020429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:17.044297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:17.047968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:17.077074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 865/1750] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:17.603061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:17.604613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:17.635632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc746b5-c4d3-45be-84d9-7b86b00774a0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:17.654024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:17.655126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:17.655813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:17.656879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] PATH_INFO: `/v3/users/f0fb165c490046e8b6044c329b3404fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:17.662556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0fb165c490046e8b6044c329b3404fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:17.668242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:17.789808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0fb165c490046e8b6044c329b3404fc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:17.809634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0fb165c490046e8b6044c329b3404fc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:17.813915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:17.840770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0fb165c490046e8b6044c329b3404fc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:17.841913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0fb165c490046e8b6044c329b3404fc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:17.844181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-08cc150b-e7c9-47f5-82ed-b3b914be703e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0fb165c490046e8b6044c329b3404fc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:17.864046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:17.865743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:17.908399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 886/1751] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:17 2022] DELETE /identity/v3/users/f0fb165c490046e8b6044c329b3404fc => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:17.919716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8130f8a1-8dd9-4732-adc9-42b415a05cd3 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:17.921104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:17.921403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:17.921680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:17.997176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:17.998756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:18.173828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:18.185759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.186059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.186353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] PATH_INFO: `/v3/projects/d1416faa95b24d76b2189a0cf79e3389` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.207737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1416faa95b24d76b2189a0cf79e3389)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:18.220521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:18.256657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 887/1752] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:18.290234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:18.292171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:18.311386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:18.421909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1416faa95b24d76b2189a0cf79e3389', 'request_id': 'req-c74f3821-22bf-4e48-bbca-17300e5028da'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:18.448005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1416faa95b24d76b2189a0cf79e3389', 'request_id': 'req-c74f3821-22bf-4e48-bbca-17300e5028da'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:18.468234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1416faa95b24d76b2189a0cf79e3389', 'request_id': 'req-c74f3821-22bf-4e48-bbca-17300e5028da'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:18.471369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd99236-bcea-454a-b48c-1625c5bef825 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:18.510084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] The token cache is being invalidate because project d1416faa95b24d76b2189a0cf79e3389 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:18.510632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:18.514247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c74f3821-22bf-4e48-bbca-17300e5028da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:18.521423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 866/1753] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:17 2022] DELETE /identity/v3/projects/d1416faa95b24d76b2189a0cf79e3389 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:18.526051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe19c7-77a7-4a0d-a7aa-4c48ad74e722 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.526580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe19c7-77a7-4a0d-a7aa-4c48ad74e722 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.527025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe19c7-77a7-4a0d-a7aa-4c48ad74e722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.567474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-69fe19c7-77a7-4a0d-a7aa-4c48ad74e722 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:18.570953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-69fe19c7-77a7-4a0d-a7aa-4c48ad74e722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:18.676961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:18.677595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.677992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.678272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.681606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:18.792198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 867/1754] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:18.792198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85e595-61b4-4c9d-99b0-a659ee97e6e0 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.792198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85e595-61b4-4c9d-99b0-a659ee97e6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.792198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f85e595-61b4-4c9d-99b0-a659ee97e6e0 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.795336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 868/1755] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:56:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:56:18.802004 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4d3f-a52c-423f-89d8-7d4281e408e9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.802285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4d3f-a52c-423f-89d8-7d4281e408e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.807762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4db4d3f-a52c-423f-89d8-7d4281e408e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.848573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:18.865459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e4db4d3f-a52c-423f-89d8-7d4281e408e9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:18.867568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e4db4d3f-a52c-423f-89d8-7d4281e408e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:18.893147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 869/1756] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:18.898245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.898693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.899864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.904443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:18.907049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:18.936481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 888/1757] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:18.937589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:18.952351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:18.955795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:18.960045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77c2b-25a0-407e-bef4-c539cb48e9a2 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:18.989225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:18.991965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:18.993714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:18.994626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:18.999978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:18.999978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.003496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.040866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 889/1758] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:19.053087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:19.068186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaeeed-ac2d-40ef-a533-f31a40a5e7e8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:19.076004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:19.077261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.077673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.078118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.079113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.082575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.220086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 870/1759] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:56:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:19.235315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 890/1760] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:19 2022] POST /identity/v3/projects => generated 373 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:19.236694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:19.268330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82de373-4839-4191-8b85-7b7874969ecb None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:19.337917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:19.463433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:19.466090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:19.474832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:19.475530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.475838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.476136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.477853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.480933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.536249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 871/1761] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:19 2022] POST /identity/v3/users => generated 287 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:19.540967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:19.554135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64972094-291f-469c-abee-680fbdffd026 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:19.567943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:19.568659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.568941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.569379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.570247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.578810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.608312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 872/1762] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:19.617899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:19.644968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5fc39-fbff-46a9-b4cc-826ac7960dde admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:19.674205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:19.674993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.675275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.675686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.676345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.678685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.693088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f51580-83ca-427d-92ca-471686bb55fa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:19.711916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a7a8c-4bc8-4e02-97f4-980f5f882f3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:19.712667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a7a8c-4bc8-4e02-97f4-980f5f882f3c service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.712952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a7a8c-4bc8-4e02-97f4-980f5f882f3c service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.713308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a7a8c-4bc8-4e02-97f4-980f5f882f3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.728581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 873/1763] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:19.729804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:19.736730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a7a8c-4bc8-4e02-97f4-980f5f882f3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.740966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a7a8c-4bc8-4e02-97f4-980f5f882f3c service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.765681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dceca2-dfe1-4d53-80db-d022d84838ba admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:19.805014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964c9a4-1520-47b3-acc8-c6012213a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:19.805014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964c9a4-1520-47b3-acc8-c6012213a0d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.805014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964c9a4-1520-47b3-acc8-c6012213a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.805014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964c9a4-1520-47b3-acc8-c6012213a0d1 admin admin] PATH_INFO: `/v3/projects/ec83c74375574d81b4c55fcaa1e601f8/users/c0ebe89791604b76871692c247b5f36b/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.809914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 891/1764] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:19.857525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964c9a4-1520-47b3-acc8-c6012213a0d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec83c74375574d81b4c55fcaa1e601f8, user_id=c0ebe89791604b76871692c247b5f36b, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:19.865503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964c9a4-1520-47b3-acc8-c6012213a0d1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:19.895670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 874/1765] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:19 2022] PUT /identity/v3/projects/ec83c74375574d81b4c55fcaa1e601f8/users/c0ebe89791604b76871692c247b5f36b/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:19.910693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460465ec-1414-4b69-92e8-9ef37e7d9757 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:19.911020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460465ec-1414-4b69-92e8-9ef37e7d9757 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:19.911318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460465ec-1414-4b69-92e8-9ef37e7d9757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:19.981797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-460465ec-1414-4b69-92e8-9ef37e7d9757 None None] MFA Rules not processed for user `c0ebe89791604b76871692c247b5f36b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:19.984298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-460465ec-1414-4b69-92e8-9ef37e7d9757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:20.111476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e2679-e78f-4a63-8f8e-3bf70c2e8490 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:20.111955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e2679-e78f-4a63-8f8e-3bf70c2e8490 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:20.112383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e2679-e78f-4a63-8f8e-3bf70c2e8490 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:20.114385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 875/1766] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:56:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:56:20.218382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 892/1767] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:20.231230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e2679-e78f-4a63-8f8e-3bf70c2e8490 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:20.233236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e17e2679-e78f-4a63-8f8e-3bf70c2e8490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:20.334537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:20.334825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:20.335104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:20.389490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:20.406715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:20.446461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:20.493599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e2679-e78f-4a63-8f8e-3bf70c2e8490 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:20.502370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:20.503054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:20.503394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:20.503611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:20.526976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:20.529018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:20.554840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 876/1768] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:20.669976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 893/1769] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:56:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:21.030814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:21.036879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:21.229163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a286aaa-13db-40b6-a8df-c2b17dbf28a3 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:21.245870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:21.246902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:21.247317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:21.247926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] PATH_INFO: `/v3/projects/993a1f1d86a741dda11fdd32c27091b2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:21.259625 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993a1f1d86a741dda11fdd32c27091b2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:21.262291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:21.274684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:21.396662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993a1f1d86a741dda11fdd32c27091b2', 'request_id': 'req-67e7dd65-045b-4b7a-a961-d81011160973'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.415639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993a1f1d86a741dda11fdd32c27091b2', 'request_id': 'req-67e7dd65-045b-4b7a-a961-d81011160973'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.437408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993a1f1d86a741dda11fdd32c27091b2', 'request_id': 'req-67e7dd65-045b-4b7a-a961-d81011160973'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.465754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] The token cache is being invalidate because project 993a1f1d86a741dda11fdd32c27091b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:21.466109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.466734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.468607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 877/1770] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:21 2022] DELETE /identity/v3/projects/993a1f1d86a741dda11fdd32c27091b2 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:21.473013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:21.490536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37a662-619b-4aa2-ba81-9307f2efb366 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:21.578729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:21.578729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:21.578729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:21.578729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] PATH_INFO: `/v3/users/c441f51e5acf46d5bd95dab9f3f2d177` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:21.582781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c441f51e5acf46d5bd95dab9f3f2d177)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:21.584950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:21.661781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c441f51e5acf46d5bd95dab9f3f2d177'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.675964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c441f51e5acf46d5bd95dab9f3f2d177'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.695154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c441f51e5acf46d5bd95dab9f3f2d177'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.696112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c441f51e5acf46d5bd95dab9f3f2d177'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.697712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c441f51e5acf46d5bd95dab9f3f2d177'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:21.716139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 894/1771] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:21 2022] DELETE /identity/v3/users/c441f51e5acf46d5bd95dab9f3f2d177 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:21.717568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:21.719657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:21.990406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e7dd65-045b-4b7a-a961-d81011160973 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:22.014724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:22.016289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:22.165191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:22.294642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3983cc-f729-4a72-9212-2036a2c06ac0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:22.300588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:22.301434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:22.301634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:22.301946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] PATH_INFO: `/v3/projects/7f5d790baaf8420a811db6bf113a9a82` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:22.303307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5d790baaf8420a811db6bf113a9a82)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:22.306298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:22.372798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:22.375570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:22.377484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:22.377949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:22.385601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:22.428368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5d790baaf8420a811db6bf113a9a82', 'request_id': 'req-660d5298-4397-4f76-955d-2c04b1e83eb0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.448858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:22.465309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5d790baaf8420a811db6bf113a9a82', 'request_id': 'req-660d5298-4397-4f76-955d-2c04b1e83eb0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.477156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5d790baaf8420a811db6bf113a9a82', 'request_id': 'req-660d5298-4397-4f76-955d-2c04b1e83eb0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.477861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:22.502031 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] The token cache is being invalidate because project 7f5d790baaf8420a811db6bf113a9a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:22.502534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.506266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.510002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 895/1772] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:22 2022] DELETE /identity/v3/projects/7f5d790baaf8420a811db6bf113a9a82 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:22.513757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:22.515242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:22.543859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 878/1773] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:22.675526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660d5298-4397-4f76-955d-2c04b1e83eb0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:22.687644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:22.688315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:22.688591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:22.688865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] PATH_INFO: `/v3/users/e160d012676e4a32a8cde98fb3db1567` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:22.693156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e160d012676e4a32a8cde98fb3db1567)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:22.695265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:22.791200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e160d012676e4a32a8cde98fb3db1567'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.804038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e160d012676e4a32a8cde98fb3db1567'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.813738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:22.821987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e160d012676e4a32a8cde98fb3db1567'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.822767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e160d012676e4a32a8cde98fb3db1567'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.823427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e05ddbb3-7a05-4a4c-956e-c86125b1c3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e160d012676e4a32a8cde98fb3db1567'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:22.845245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 896/1774] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:22 2022] DELETE /identity/v3/users/e160d012676e4a32a8cde98fb3db1567 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:23.978133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:23.982032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:24.059039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97506bd0-8c2c-46c9-9cdb-e3d528422ae8 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:24.109912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:24.221756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:24.222662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:24.223112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:24.223577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] PATH_INFO: `/v3/projects/aeee51439ef24c45b2191fc62bbee14a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:24.224949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeee51439ef24c45b2191fc62bbee14a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:24.227310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:24.387835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeee51439ef24c45b2191fc62bbee14a', 'request_id': 'req-fb1e25f8-6593-4937-86d9-27f250e93c83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.400902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeee51439ef24c45b2191fc62bbee14a', 'request_id': 'req-fb1e25f8-6593-4937-86d9-27f250e93c83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.417497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeee51439ef24c45b2191fc62bbee14a', 'request_id': 'req-fb1e25f8-6593-4937-86d9-27f250e93c83'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.432535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] The token cache is being invalidate because project aeee51439ef24c45b2191fc62bbee14a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:24.433194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.434747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-fb1e25f8-6593-4937-86d9-27f250e93c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.437218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 879/1775] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:23 2022] DELETE /identity/v3/projects/aeee51439ef24c45b2191fc62bbee14a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:24.442683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:24.443657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:24.444211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:24.468559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:24.469818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:24.602252 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 897/1776] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:24.610423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:24.610836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:24.611157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:24.645062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:24.646629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:24.659158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:24.661544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:24.712000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 880/1777] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:24.716180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:24.718466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead47b21-3a5f-418e-9977-32fd26363dcf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:24.734302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:24.736685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:24.738865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:24.739344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] PATH_INFO: `/v3/projects/5952842d022643c9b8abfbf36fd0a205` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:24.743885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5952842d022643c9b8abfbf36fd0a205)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:24.744431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d8637-b145-4c7e-9830-511e6736e9c2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:24.749866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:24.758165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:24.758791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:24.759070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:24.759332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:24.760076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:24.760335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:24.761953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:24.783325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 881/1778] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:24.788702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:24.809854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ee12a5-9a80-4136-8927-7e06ae909cc9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:24.814910 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:24.815816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:24.816031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:24.816358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:24.817650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:24.819606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:24.877744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 882/1779] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:24 2022] POST /identity/v3/projects => generated 407 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:24.884773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:24.904020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5952842d022643c9b8abfbf36fd0a205', 'request_id': 'req-95e59f65-c209-43e8-9051-02d8395c61b2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.924356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5952842d022643c9b8abfbf36fd0a205', 'request_id': 'req-95e59f65-c209-43e8-9051-02d8395c61b2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.944684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5952842d022643c9b8abfbf36fd0a205', 'request_id': 'req-95e59f65-c209-43e8-9051-02d8395c61b2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:24.997622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] The token cache is being invalidate because project 5952842d022643c9b8abfbf36fd0a205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:24.997931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:25.011477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-95e59f65-c209-43e8-9051-02d8395c61b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:25.011477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 898/1780] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:24 2022] DELETE /identity/v3/projects/5952842d022643c9b8abfbf36fd0a205 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:25.013041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb9305-c38b-47cf-8e50-2dcb95855335 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.013333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb9305-c38b-47cf-8e50-2dcb95855335 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.013610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fb9305-c38b-47cf-8e50-2dcb95855335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.028690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639c693-93e6-4243-a165-ff5c990534c3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.040385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.041722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.042527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.043150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.044532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.046919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.061330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:25.089487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 883/1781] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:24 2022] POST /identity/v3/users => generated 304 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:25.091538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.109323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:25.110365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-75fb9305-c38b-47cf-8e50-2dcb95855335 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:25.111602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-75fb9305-c38b-47cf-8e50-2dcb95855335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:25.139358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7a8857-6eac-4c75-ad22-93bf3db2d0de admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.157273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.158281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.158800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.159845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.160759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.165776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.167372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 899/1782] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:25.168988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.169279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.176706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.194802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 884/1783] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:25.198328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.219924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdae01f-ed11-484f-93d8-d5abc2373e10 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.227377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:25.231265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:25.232698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.233966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.234531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.235258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.236165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.238409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.259505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 885/1784] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:25 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:25.261073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.264081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 900/1785] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:25.268013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.280879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467930bd-d0c7-4617-965c-286166c2d601 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.297897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dab7233-ed81-4a44-8fa7-5082494e2fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.299506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab7233-ed81-4a44-8fa7-5082494e2fb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.300933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab7233-ed81-4a44-8fa7-5082494e2fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.302511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dab7233-ed81-4a44-8fa7-5082494e2fb8 admin admin] PATH_INFO: `/v3/projects/3a9fa2b25ee449c5a5a3206766a4bec2/users/5d595d4be1284a63b4d393bfae951853/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.303574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb8f67-3623-4a2b-a9c1-a0932c7befeb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.327802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.328482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.328767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.329044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.330000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:25.330309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.332042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.343977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab7233-ed81-4a44-8fa7-5082494e2fb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a9fa2b25ee449c5a5a3206766a4bec2, user_id=5d595d4be1284a63b4d393bfae951853, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.348727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dab7233-ed81-4a44-8fa7-5082494e2fb8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.382905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 901/1786] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:25.387784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.402871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 886/1787] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:25 2022] PUT /identity/v3/projects/3a9fa2b25ee449c5a5a3206766a4bec2/users/5d595d4be1284a63b4d393bfae951853/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:25.407184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.407758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.408409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.418595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d332d38f-2edd-44d8-b425-abc69ce84c1e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.510973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] MFA Rules not processed for user `5d595d4be1284a63b4d393bfae951853`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:25.514891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:25.532958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:25.605086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.605841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.605999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.606280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.606928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.608857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.651183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 887/1788] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:25 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:25.652322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.656853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:25.678543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 902/1789] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:25 2022] POST /identity/v3/projects => generated 365 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:25.679841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.693736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a438fde-4fe2-4b60-a7dc-6fa1400d5842 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.791791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d96eb4-6fa3-4be8-919e-81926f1909ba admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:25.797582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.798143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.798423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.798696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.800474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.802586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.845525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:25.846193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:25.849359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:25.849644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] PATH_INFO: `/v3/users/c0ebe89791604b76871692c247b5f36b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:25.851982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0ebe89791604b76871692c247b5f36b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:25.856241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:25.863856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 903/1790] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:25 2022] POST /identity/v3/projects => generated 407 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:25.864753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:25.952350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:25.968871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0ebe89791604b76871692c247b5f36b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:25.985170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0ebe89791604b76871692c247b5f36b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:25.986454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0ebe89791604b76871692c247b5f36b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:26.017264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00649f3-44a3-493b-a205-2c13baa9b5ce admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.023442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0ebe89791604b76871692c247b5f36b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:26.024495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558c488e-74c4-4f02-aff2-24fb2d8a24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.025447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c488e-74c4-4f02-aff2-24fb2d8a24f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.025447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c488e-74c4-4f02-aff2-24fb2d8a24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.025780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558c488e-74c4-4f02-aff2-24fb2d8a24f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.026676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c488e-74c4-4f02-aff2-24fb2d8a24f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.026778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0ebe89791604b76871692c247b5f36b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:26.029003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558c488e-74c4-4f02-aff2-24fb2d8a24f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.062002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 888/1791] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:25 2022] DELETE /identity/v3/users/c0ebe89791604b76871692c247b5f36b => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:26.062758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 904/1792] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:25 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:26.070597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.071718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:26.073169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.073624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.074265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.114182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:26.115445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:26.203636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:26.229117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4ea72-7b1e-423e-820c-5cc7231ee8dc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.236371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.237901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.238639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.238885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.239872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.242388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.253882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 905/1793] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:26.255256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.265575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:26.279850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d378d0-c5f3-4559-bc23-b14a5a388932 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.280848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 889/1794] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:26 2022] POST /identity/v3/users => generated 310 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:26.282082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.288237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.288812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.289093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.289272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:26.289370 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.290014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.291667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.304724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 906/1795] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:26.305438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.323751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aac37e-3b2d-48d2-9d78-4d80fa277237 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.329342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.329921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.330202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.330483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.331162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.333014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.356834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 907/1796] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:26.357971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.382183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7366b-5d5a-4b77-ac3f-211da6ed7f0a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.389829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ca67-691c-4755-b2b2-a8236ccf9aa6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.391027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.393056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.394461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.395089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.395464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.395810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.396936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.397639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.400353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.404913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.406765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.408657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.431869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 890/1797] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:26.437032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.442301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 908/1798] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:26.443203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.453647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db3eeb7-6e3a-4119-8e4e-b71fe08e5f90 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.460159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.461000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.461192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.461472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] PATH_INFO: `/v3/projects/b4ed96f51c414eee9abb40f068ff71c4/users/23027aaf70a1428f872f895a425e9915/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.467453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405355bd-43a8-47af-8774-b216750142bb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.477901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858757c-41f8-4d37-b994-981687a6e3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.478607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858757c-41f8-4d37-b994-981687a6e3a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.478897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858757c-41f8-4d37-b994-981687a6e3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.479241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858757c-41f8-4d37-b994-981687a6e3a7 admin admin] PATH_INFO: `/v3/projects/3881f18b697e4627aa4282fa2e3a0b49/users/4bcdf74e348e4ea297590c0709ce13af/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.508044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ed96f51c414eee9abb40f068ff71c4, user_id=23027aaf70a1428f872f895a425e9915, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.517423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.524910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858757c-41f8-4d37-b994-981687a6e3a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3881f18b697e4627aa4282fa2e3a0b49, user_id=4bcdf74e348e4ea297590c0709ce13af, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.528045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858757c-41f8-4d37-b994-981687a6e3a7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.544472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 891/1799] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:26 2022] PUT /identity/v3/projects/b4ed96f51c414eee9abb40f068ff71c4/users/23027aaf70a1428f872f895a425e9915/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:26.548630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.594677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 909/1800] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:26 2022] PUT /identity/v3/projects/3881f18b697e4627aa4282fa2e3a0b49/users/4bcdf74e348e4ea297590c0709ce13af/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:26.597872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.598357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.598713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.615180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:26.714334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] MFA Rules not processed for user `4bcdf74e348e4ea297590c0709ce13af`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:26.716314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:26.717453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d123-0e7c-4111-8191-47df0c70c60e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.881743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.883037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.883644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.885239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.886869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.891792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:26.914319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 892/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:26.918748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:26.949971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894f011-47c0-4a09-b0f5-7a42e0123fc2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:26.965803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5934e-0fd2-4c0d-96b2-7170f8b66194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:26.966879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5934e-0fd2-4c0d-96b2-7170f8b66194 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:26.967389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5934e-0fd2-4c0d-96b2-7170f8b66194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:26.967891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5934e-0fd2-4c0d-96b2-7170f8b66194 admin admin] PATH_INFO: `/v3/projects/b4ed96f51c414eee9abb40f068ff71c4/users/23027aaf70a1428f872f895a425e9915/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:26.973624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5934e-0fd2-4c0d-96b2-7170f8b66194 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ed96f51c414eee9abb40f068ff71c4, user_id=23027aaf70a1428f872f895a425e9915, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:26.977536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5934e-0fd2-4c0d-96b2-7170f8b66194 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.006590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 893/1802] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:26 2022] PUT /identity/v3/projects/b4ed96f51c414eee9abb40f068ff71c4/users/23027aaf70a1428f872f895a425e9915/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:27.009991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.010722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.010950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.046457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 910/1803] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:27.047799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.054007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] MFA Rules not processed for user `23027aaf70a1428f872f895a425e9915`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:27.055563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:27.074489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1afaf-6c6a-48cf-8b31-1c005b66b05e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.163607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 894/1804] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:27.201009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.208133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:27.240764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.245822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.246323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.246802 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.247726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.253133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.294932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:27.296542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:27.316618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 911/1805] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:27 2022] POST /identity/v3/projects => generated 365 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:27.320708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.353244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68dbeb3-4024-4785-b8fc-2fef8251fbb0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.431827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.432700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.432891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.433153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.463586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e3631c-f30b-4ecf-accc-51b6bc8bd36f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.477678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.479104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.480711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.481233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.482083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.484749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.530292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.531876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.533774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 912/1806] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:27 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:27.537697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.549138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 895/1807] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:27.571706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb04ee3-3368-47de-ba95-108307071553 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.586758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.588119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.588517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.588902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.590181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.592157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.609349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 913/1808] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:27.609990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.624402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284aa56b-1af6-4020-a608-42e25b717772 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.635616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.635616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.635853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.636036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] PATH_INFO: `/v3/projects/f5df3077ae254593a16b2f4d03e3f7b2/users/dac3058d5fc74646a7c65ba7f45f67aa/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.655656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5df3077ae254593a16b2f4d03e3f7b2, user_id=dac3058d5fc74646a7c65ba7f45f67aa, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.658477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.679813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 896/1809] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:27 2022] PUT /identity/v3/projects/f5df3077ae254593a16b2f4d03e3f7b2/users/dac3058d5fc74646a7c65ba7f45f67aa/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:27.682062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.695283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea84d14d-b023-4daa-b62d-3b01e4809964 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.726866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.730987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:27.790036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.790740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.790944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.791223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.791941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.793764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.810114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 914/1810] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:27 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:27.816423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b944c18-83bb-43fb-bb4a-730e2cf4f081 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.816662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.821318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.822008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.822289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.822562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] PATH_INFO: `/v3/projects/ec83c74375574d81b4c55fcaa1e601f8` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.825994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec83c74375574d81b4c55fcaa1e601f8)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.829317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.829924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471378f-dea7-4f81-864e-87d96dffb132 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.836088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.836995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.837393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.837740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] PATH_INFO: `/v3/projects/f5df3077ae254593a16b2f4d03e3f7b2/users/dac3058d5fc74646a7c65ba7f45f67aa/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.840477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5df3077ae254593a16b2f4d03e3f7b2, user_id=dac3058d5fc74646a7c65ba7f45f67aa, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:27.843207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:27.859051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 915/1811] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:27 2022] PUT /identity/v3/projects/f5df3077ae254593a16b2f4d03e3f7b2/users/dac3058d5fc74646a7c65ba7f45f67aa/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:27.863630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:27.869261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:27.892745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec83c74375574d81b4c55fcaa1e601f8', 'request_id': 'req-b1261398-8867-4f70-8b01-507614d5e60f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:27.902000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec83c74375574d81b4c55fcaa1e601f8', 'request_id': 'req-b1261398-8867-4f70-8b01-507614d5e60f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:27.909290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec83c74375574d81b4c55fcaa1e601f8', 'request_id': 'req-b1261398-8867-4f70-8b01-507614d5e60f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:27.921144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] The token cache is being invalidate because project ec83c74375574d81b4c55fcaa1e601f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:27.921439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:27.922130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1261398-8867-4f70-8b01-507614d5e60f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:27.923727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 897/1812] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:27 2022] DELETE /identity/v3/projects/ec83c74375574d81b4c55fcaa1e601f8 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:27.925909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1787d-5f60-4dc3-ba08-c7650852887e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.926050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1787d-5f60-4dc3-ba08-c7650852887e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.926225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1787d-5f60-4dc3-ba08-c7650852887e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.936501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c7c0d7-8b75-4a3f-97fd-13fd5d0ebf66 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:27.948874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:27.958448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0cf1787d-5f60-4dc3-ba08-c7650852887e None None] MFA Rules not processed for user `dac3058d5fc74646a7c65ba7f45f67aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:27.959706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0cf1787d-5f60-4dc3-ba08-c7650852887e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:27.992493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:27.993104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:27.993383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:27.993663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:27.995384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:28.029694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 898/1813] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:28.030941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.031228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.031495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.053513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:28.054713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:28.056813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.058257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.067868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 916/1814] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:27 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:28.068739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.070331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:28.080353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24155f4-740e-499e-87c9-874fc68881bf service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.084787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae890f39-948c-4351-952d-50a69366cae2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.085514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae890f39-948c-4351-952d-50a69366cae2 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.085865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae890f39-948c-4351-952d-50a69366cae2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.086151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae890f39-948c-4351-952d-50a69366cae2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.097830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae890f39-948c-4351-952d-50a69366cae2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.099264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae890f39-948c-4351-952d-50a69366cae2 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.110134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 917/1815] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:28.125892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 899/1816] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:28.127089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.127089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.127386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.150100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:28.151360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:28.167747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 918/1817] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:28.171937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.183515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3767b2-5f44-42da-b12c-41090a8471ca None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.188602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.189125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.189405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.189670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.190428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:28.190732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.192415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.212940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 900/1818] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:28.217018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.227640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f523e188-646d-4b07-bf97-3c5e6e765bec None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.232496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.235774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.235774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.235774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.235774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.236264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.275589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 919/1819] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:28 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:28.278494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.290032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2b762-de09-406d-8694-4ac6196e46c5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.325328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:28.417398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.418136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.418317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.418590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.419241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.421482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.466598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 901/1820] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:28 2022] POST /identity/v3/users => generated 301 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:28.470431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.485028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cebfbe-26c5-4d0e-807b-8b6b49b737e7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.494734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.495445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.496096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.497535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.498426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.502312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.528104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 920/1821] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:28.532477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.561797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb02f5cc-1c7f-4795-8807-333c8d19943c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.579808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58f1d46-fd10-4e94-bab7-16a01d0abd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.580510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1d46-fd10-4e94-bab7-16a01d0abd8e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.581538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1d46-fd10-4e94-bab7-16a01d0abd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.582069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58f1d46-fd10-4e94-bab7-16a01d0abd8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.583257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f1d46-fd10-4e94-bab7-16a01d0abd8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.586342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58f1d46-fd10-4e94-bab7-16a01d0abd8e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.609976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 902/1822] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:28.615618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.630485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270b6c94-bb35-41bd-8bec-4aa02dc67295 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.636596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.637285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.637563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.637835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] PATH_INFO: `/v3/projects/66f043feb09f4585af419e21a9566c54/users/bba0e831cc414c9c939fd334eda4f601/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.654995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f043feb09f4585af419e21a9566c54, user_id=bba0e831cc414c9c939fd334eda4f601, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.658491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.676061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 921/1823] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:28 2022] PUT /identity/v3/projects/66f043feb09f4585af419e21a9566c54/users/bba0e831cc414c9c939fd334eda4f601/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:28.681162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.681453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.681721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.701974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:28.703113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:28.780056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 903/1824] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:28.788242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:28.790220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:28.868223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961ec7-01e2-42c9-80ca-e7e13c71fd88 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:28.873790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c421a-bc7c-4ec1-9d2b-e1c035b0a2db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:28.874071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c421a-bc7c-4ec1-9d2b-e1c035b0a2db service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:28.874359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c421a-bc7c-4ec1-9d2b-e1c035b0a2db service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:28.874638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c421a-bc7c-4ec1-9d2b-e1c035b0a2db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:28.882989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:28.890037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c421a-bc7c-4ec1-9d2b-e1c035b0a2db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:28.892863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7c421a-bc7c-4ec1-9d2b-e1c035b0a2db service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:28.918266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 922/1825] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:29.067639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:29.069116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:29.082743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872180f3-a7cb-47cc-923f-cc8122e3e21a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:29.087928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:29.088633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:29.089004 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:29.089350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] PATH_INFO: `/v3/users/4bcdf74e348e4ea297590c0709ce13af` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:29.090629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bcdf74e348e4ea297590c0709ce13af)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:29.092711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:29.151417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bcdf74e348e4ea297590c0709ce13af'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:29.165465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bcdf74e348e4ea297590c0709ce13af'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:29.166133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bcdf74e348e4ea297590c0709ce13af'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:29.181078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bcdf74e348e4ea297590c0709ce13af'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:29.188127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bcdf74e348e4ea297590c0709ce13af'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:29.202825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 904/1826] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:29 2022] DELETE /identity/v3/users/4bcdf74e348e4ea297590c0709ce13af => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:29.207297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c08101-5804-42d6-99b0-1c84755fcd6d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:29.207708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c08101-5804-42d6-99b0-1c84755fcd6d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:29.208080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c08101-5804-42d6-99b0-1c84755fcd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:29.231096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-02c08101-5804-42d6-99b0-1c84755fcd6d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:29.231096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-02c08101-5804-42d6-99b0-1c84755fcd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:29.301170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 923/1827] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:29.342367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:29.343935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:29.368852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:29.449452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72b2388-3c33-4549-b350-fbc0dcc31c82 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:29.453969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:29.454742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:29.455107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:29.455446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:29.468565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:29.471108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:29.486164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 905/1828] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:30.052303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.052827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.052949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.104131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] MFA Rules not processed for user `bba0e831cc414c9c939fd334eda4f601`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:30.104902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.166631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.168519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.189049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e8d90-8d59-4ef3-8ef6-643b548150c5 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.196297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.196948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.197338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.197488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] PATH_INFO: `/v3/projects/3881f18b697e4627aa4282fa2e3a0b49` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.200280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3881f18b697e4627aa4282fa2e3a0b49)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.202229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.259717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 924/1829] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:30.278658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.284659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.305727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3881f18b697e4627aa4282fa2e3a0b49', 'request_id': 'req-369d9825-822f-449a-8b1b-9ac7a7edfc6a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.322255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3881f18b697e4627aa4282fa2e3a0b49', 'request_id': 'req-369d9825-822f-449a-8b1b-9ac7a7edfc6a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.325347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c06edf-a252-4b89-9cc2-1000d41e3c74 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.344333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3881f18b697e4627aa4282fa2e3a0b49', 'request_id': 'req-369d9825-822f-449a-8b1b-9ac7a7edfc6a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.362131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] The token cache is being invalidate because project 3881f18b697e4627aa4282fa2e3a0b49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:30.362421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.363279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.366992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 906/1830] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:30 2022] DELETE /identity/v3/projects/3881f18b697e4627aa4282fa2e3a0b49 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:30.371684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.374157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.410391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.411161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.411442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.411803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.412443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.414625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.448134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 925/1831] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:30 2022] POST /identity/v3/projects => generated 405 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:30.453429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.454403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.461807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369d9825-822f-449a-8b1b-9ac7a7edfc6a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.465584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:30.514929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:30.525166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ce2654-fe00-4028-a621-a9f838012601 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.526975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.527638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.527948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.528216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] PATH_INFO: `/v3/users/dac3058d5fc74646a7c65ba7f45f67aa` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.529355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac3058d5fc74646a7c65ba7f45f67aa)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.529864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.530476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.530757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.531032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.531257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.531703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.534530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.556867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 926/1832] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:30 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:30.557804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.560290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.573102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af863225-e0d6-4d0e-ac3b-06ee0f1daa56 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.577699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.578321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.578610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.578888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] PATH_INFO: `/v3/users/431d40a24b3e4ecdb0b5154543ec515f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.580047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] RBAC: Authorizing `identity:delete_user(user_id=431d40a24b3e4ecdb0b5154543ec515f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.581172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac3058d5fc74646a7c65ba7f45f67aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.582094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.589620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac3058d5fc74646a7c65ba7f45f67aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.590514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac3058d5fc74646a7c65ba7f45f67aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.603500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac3058d5fc74646a7c65ba7f45f67aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.604188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac3058d5fc74646a7c65ba7f45f67aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.619302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 907/1833] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:30 2022] DELETE /identity/v3/users/dac3058d5fc74646a7c65ba7f45f67aa => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:30.620331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.623920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.635896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '431d40a24b3e4ecdb0b5154543ec515f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.644629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '431d40a24b3e4ecdb0b5154543ec515f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.660796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '431d40a24b3e4ecdb0b5154543ec515f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.661569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '431d40a24b3e4ecdb0b5154543ec515f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.662196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-300f91d8-2b01-456a-90f5-45b1af12150f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '431d40a24b3e4ecdb0b5154543ec515f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:30.679811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 927/1834] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:30 2022] DELETE /identity/v3/users/431d40a24b3e4ecdb0b5154543ec515f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:30.684392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.684801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.685078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.699365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f949705d-ad8b-4c39-a210-ff02033c3bd1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.707375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:30.708775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.771050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.771701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.771982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.772258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.772907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.774505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.786858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 928/1835] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:30.788636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 908/1836] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:30.789900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.791427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.797310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.800332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.807001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f0239-bd11-43e5-9fbf-d41c7425bb13 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.813816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.814618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.814803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:30.814946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.815226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] PATH_INFO: `/v3/projects/636b87793ba84a8e9bd5c76833c5aa8f/users/1472c375a22e4323b7836b11b1e975aa/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.826590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:30.835268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636b87793ba84a8e9bd5c76833c5aa8f, user_id=1472c375a22e4323b7836b11b1e975aa, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.838772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.854588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 929/1837] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:30 2022] PUT /identity/v3/projects/636b87793ba84a8e9bd5c76833c5aa8f/users/1472c375a22e4323b7836b11b1e975aa/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:30.858602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.869087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b8024-09ab-450d-bdae-e3edc46cab1f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.934106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a9ca30-b11d-402c-b7ac-9c9b87928972 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.936006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.936632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.937117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.937535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.943081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.943965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.944418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.944694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.945345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.946992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.950719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.952206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.967735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 930/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:30.970292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.972941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:30.982536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 909/1839] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:30.983246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:30.988966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a363cdd6-66cf-4985-9863-0cb62dcb7333 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.993432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a985f3-e2f8-4b99-b23f-f9fb7a94c926 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:30.993951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.994688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.995064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.995448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] PATH_INFO: `/v3/projects/f5df3077ae254593a16b2f4d03e3f7b2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:30.996681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5df3077ae254593a16b2f4d03e3f7b2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:30.998350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95244b6-3a23-4150-b31b-75cd1ed1e9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:30.999372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:30.999485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95244b6-3a23-4150-b31b-75cd1ed1e9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:30.999485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95244b6-3a23-4150-b31b-75cd1ed1e9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:30.999709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95244b6-3a23-4150-b31b-75cd1ed1e9d1 admin admin] PATH_INFO: `/v3/projects/636b87793ba84a8e9bd5c76833c5aa8f/users/1472c375a22e4323b7836b11b1e975aa/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.001616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95244b6-3a23-4150-b31b-75cd1ed1e9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=636b87793ba84a8e9bd5c76833c5aa8f, user_id=1472c375a22e4323b7836b11b1e975aa, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.004368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95244b6-3a23-4150-b31b-75cd1ed1e9d1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.023961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 910/1840] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:30 2022] PUT /identity/v3/projects/636b87793ba84a8e9bd5c76833c5aa8f/users/1472c375a22e4323b7836b11b1e975aa/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:31.061942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5df3077ae254593a16b2f4d03e3f7b2', 'request_id': 'req-f006fed4-d0bf-4bda-a1a9-edb109938c3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.075787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5df3077ae254593a16b2f4d03e3f7b2', 'request_id': 'req-f006fed4-d0bf-4bda-a1a9-edb109938c3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.085430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5df3077ae254593a16b2f4d03e3f7b2', 'request_id': 'req-f006fed4-d0bf-4bda-a1a9-edb109938c3a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.102734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] The token cache is being invalidate because project f5df3077ae254593a16b2f4d03e3f7b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:31.110811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.112483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f006fed4-d0bf-4bda-a1a9-edb109938c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.115675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 931/1841] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:30 2022] DELETE /identity/v3/projects/f5df3077ae254593a16b2f4d03e3f7b2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:31.121984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.122353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.122731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.147496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:31.149057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:31.249377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 911/1842] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:31.255773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.256110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.256474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.296893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:31.298850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:31.310459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 932/1843] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:31.315702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.326464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400f855a-ff70-4a91-bd21-abdb981fce25 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.342793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.343520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.343981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.344406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.345169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:31.345568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.350155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.371954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 912/1844] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:31.375460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.388022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4eae50-3cde-4060-b761-5ed257ea600f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.397725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.398744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.399169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.400946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.402173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.405403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.460891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 933/1845] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:31 2022] POST /identity/v3/projects => generated 401 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:31.465035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.479790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6df1faf-48af-4954-84ef-b59cd227ca40 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.521383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.523408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:31.591374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:31.616732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:31.636771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.637742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.638030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.640270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.641457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.646329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.659365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9d72-e521-4e26-a289-af9d51761650 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.670026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.670827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.672976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.672976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] PATH_INFO: `/v3/projects/eb601b43b8cf4480839d87b7eb24d1b7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.677382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb601b43b8cf4480839d87b7eb24d1b7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.682006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.701012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 913/1846] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:31 2022] POST /identity/v3/users => generated 301 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:31.705171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.727068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54f87f0-f420-4b1c-9595-06c3cbefa611 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.732876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.733650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.733940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.734265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.734964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.736676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.763566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 914/1847] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:31.773067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.779203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb601b43b8cf4480839d87b7eb24d1b7', 'request_id': 'req-8997c0f3-d105-43b0-97cf-4e82d24d00e4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.788657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d5bf0-555c-430c-b3d7-e84edbee97a6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.789148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb601b43b8cf4480839d87b7eb24d1b7', 'request_id': 'req-8997c0f3-d105-43b0-97cf-4e82d24d00e4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.797760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb601b43b8cf4480839d87b7eb24d1b7', 'request_id': 'req-8997c0f3-d105-43b0-97cf-4e82d24d00e4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.809794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] The token cache is being invalidate because project eb601b43b8cf4480839d87b7eb24d1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:31.810083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.810766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.812436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 934/1848] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:31 2022] DELETE /identity/v3/projects/eb601b43b8cf4480839d87b7eb24d1b7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:31.816958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.818331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:31.857598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.858204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.858490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.858864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.859575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.861118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.875507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 915/1849] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:31.876915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:31.879519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:31.893338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdd5f7-1f06-4e0f-936f-4dbf77439e14 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.897123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997c0f3-d105-43b0-97cf-4e82d24d00e4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:31.897532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c299db4a-bce8-4bfc-9f16-d7ebf3ce7395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.898188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299db4a-bce8-4bfc-9f16-d7ebf3ce7395 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.898466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299db4a-bce8-4bfc-9f16-d7ebf3ce7395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.898736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c299db4a-bce8-4bfc-9f16-d7ebf3ce7395 admin admin] PATH_INFO: `/v3/projects/06fface65be44d029bf953b65d2ab7f7/users/1831430fc92f447d931ed32f83872f48/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.902550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:31.903160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.903537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.904001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] PATH_INFO: `/v3/users/91ea2ffacbaa4c43b7c5cf6c6cfb1924` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.905132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ea2ffacbaa4c43b7c5cf6c6cfb1924)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.907206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.916537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299db4a-bce8-4bfc-9f16-d7ebf3ce7395 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06fface65be44d029bf953b65d2ab7f7, user_id=1831430fc92f447d931ed32f83872f48, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:31.919439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c299db4a-bce8-4bfc-9f16-d7ebf3ce7395 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:31.937475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 916/1850] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:31 2022] PUT /identity/v3/projects/06fface65be44d029bf953b65d2ab7f7/users/1831430fc92f447d931ed32f83872f48/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:31.950326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:31.950786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:31.951215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:31.980345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ea2ffacbaa4c43b7c5cf6c6cfb1924'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.991610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ea2ffacbaa4c43b7c5cf6c6cfb1924'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:31.991815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] MFA Rules not processed for user `1831430fc92f447d931ed32f83872f48`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:31.993658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.009719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:32.012045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ea2ffacbaa4c43b7c5cf6c6cfb1924'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.013181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ea2ffacbaa4c43b7c5cf6c6cfb1924'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.014140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bca4fb9f-f4da-4db7-82c8-9e3c5c9b58ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ea2ffacbaa4c43b7c5cf6c6cfb1924'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.030481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 935/1851] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:31 2022] DELETE /identity/v3/users/91ea2ffacbaa4c43b7c5cf6c6cfb1924 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:32.032052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.032596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.033010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.078137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] MFA Rules not processed for user `1472c375a22e4323b7836b11b1e975aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:32.079505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.083016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 917/1852] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:32.091209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.092870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.147265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:32.167437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 936/1853] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:32.205363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d032cad9-e25d-4b9b-9084-cae0939d2a29 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.288726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.289363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.289726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.290012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.290813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.293046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.327291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 918/1854] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:32 2022] POST /identity/v3/projects => generated 401 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:32.331930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.346890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea21d44-24fb-4343-b0f0-0deff845f702 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.418238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.419597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.419597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.419597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.420368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.422510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.454637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 937/1855] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:32 2022] POST /identity/v3/users => generated 307 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:32.459487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.472488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceda4a2-c41f-4b52-88a9-de06f8fa6e74 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.479436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.480306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.480833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.481190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.481999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.483936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.497562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 919/1856] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:32.503241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.510545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.515561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a6222-104a-4d30-9732-b55cad4c869f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.524219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.554977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.555707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.555986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.556259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] PATH_INFO: `/v3/projects/6e856e39d29a456881fae740b904d2ef/users/bad3996187684897a4f5d89823006511/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.558881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c7399-a892-48f5-aef4-ac47bb4fe1d2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.564458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.565234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.565603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.565942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] PATH_INFO: `/v3/projects/0def520da8144582a9db2b3dfde22896` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.567152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0def520da8144582a9db2b3dfde22896)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.569164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.578058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e856e39d29a456881fae740b904d2ef, user_id=bad3996187684897a4f5d89823006511, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.581559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.595991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 938/1857] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:32 2022] PUT /identity/v3/projects/6e856e39d29a456881fae740b904d2ef/users/bad3996187684897a4f5d89823006511/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:32.600019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.629188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0def520da8144582a9db2b3dfde22896', 'request_id': 'req-a3c906ba-a433-43de-8111-e146aaa9e0e9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.637210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0def520da8144582a9db2b3dfde22896', 'request_id': 'req-a3c906ba-a433-43de-8111-e146aaa9e0e9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.645141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0def520da8144582a9db2b3dfde22896', 'request_id': 'req-a3c906ba-a433-43de-8111-e146aaa9e0e9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.653750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:32.657299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] The token cache is being invalidate because project 0def520da8144582a9db2b3dfde22896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:32.657630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.658324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a3c906ba-a433-43de-8111-e146aaa9e0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:32.660729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 920/1858] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:32 2022] DELETE /identity/v3/projects/0def520da8144582a9db2b3dfde22896 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:32.666432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bacd74-9317-4f1a-b3a3-60f4fa76e496 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.666714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bacd74-9317-4f1a-b3a3-60f4fa76e496 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.666984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bacd74-9317-4f1a-b3a3-60f4fa76e496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.682175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fb5dd-4902-4fbe-87b0-7db7904ea2d9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.688500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e5bacd74-9317-4f1a-b3a3-60f4fa76e496 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:32.689733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e5bacd74-9317-4f1a-b3a3-60f4fa76e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.733214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:32.764973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.765527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.765809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.766088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.766776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.769080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.792120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 939/1859] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:32.802104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 921/1860] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:32.802311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.802311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.804780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44565936-5081-429d-8be8-434a85a10888 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.805233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44565936-5081-429d-8be8-434a85a10888 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.805609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44565936-5081-429d-8be8-434a85a10888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.837070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3831865-787d-44f6-b875-bbb009104009 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.843506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-44565936-5081-429d-8be8-434a85a10888 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:32.843506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-44565936-5081-429d-8be8-434a85a10888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.843813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872b3b5-c76d-4c29-a00c-3dfebfd20eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.843813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872b3b5-c76d-4c29-a00c-3dfebfd20eb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.843813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872b3b5-c76d-4c29-a00c-3dfebfd20eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.843813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872b3b5-c76d-4c29-a00c-3dfebfd20eb1 admin admin] PATH_INFO: `/v3/projects/6e856e39d29a456881fae740b904d2ef/users/bad3996187684897a4f5d89823006511/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.851965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872b3b5-c76d-4c29-a00c-3dfebfd20eb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e856e39d29a456881fae740b904d2ef, user_id=bad3996187684897a4f5d89823006511, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.858479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872b3b5-c76d-4c29-a00c-3dfebfd20eb1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:32.887962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 922/1861] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:32.891385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44565936-5081-429d-8be8-434a85a10888 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:32.898206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 940/1862] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:32 2022] PUT /identity/v3/projects/6e856e39d29a456881fae740b904d2ef/users/bad3996187684897a4f5d89823006511/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:32.902053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44565936-5081-429d-8be8-434a85a10888 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:32.913716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.914118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.914537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.966818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] MFA Rules not processed for user `bad3996187684897a4f5d89823006511`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:32.968355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:32.995674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:32.996297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:32.996575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:32.996846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:32.997504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:32.997812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:32.999396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.015573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 923/1863] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:33.016589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.027322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d565d11-ff54-4905-9147-0e811ddd22f3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.032997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.033627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.033880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.034155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.034813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.036749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.072251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 924/1864] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:33 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:33.073649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 941/1865] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:32 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:33.074556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.081550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.083020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:33.145200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9accb92a-f57e-46e4-a7a3-177a388f5b95 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.146129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd79bb-535f-4e4b-bd88-2f4af5c6d382 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.150215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:33.150770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.151402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.151872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.152290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.152923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.154845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.179275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 925/1866] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:33 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:33.184504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.194317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2f2061-a439-4114-882e-2f2d31c37a78 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.199167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.199800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.200081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.200355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.201063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.202755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.209449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb99bbc-ed49-43ba-a9f1-e519b58a7089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.210066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb99bbc-ed49-43ba-a9f1-e519b58a7089 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.210347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb99bbc-ed49-43ba-a9f1-e519b58a7089 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.210619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb99bbc-ed49-43ba-a9f1-e519b58a7089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.213946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 926/1867] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:33.218124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.228491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe5582-7eb7-442a-bc19-ab4bc1aed788 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.233327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.234145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.234623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.234968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.235708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.237918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.248610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 927/1868] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:33.252647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.258583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:33.263346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc4fad-bbef-44bc-93ed-6a47b8fc251b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.269931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.270547 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.270944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.271106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] PATH_INFO: `/v3/projects/2c8664e7ed9d45c6979d5730354fd429/users/c5cfcaa270604f7ea0b7a07a63968a7a/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.276494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb99bbc-ed49-43ba-a9f1-e519b58a7089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.278051 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb99bbc-ed49-43ba-a9f1-e519b58a7089 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.286468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c8664e7ed9d45c6979d5730354fd429, user_id=c5cfcaa270604f7ea0b7a07a63968a7a, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.288476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 942/1869] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:33.289175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.303282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 928/1870] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:33 2022] PUT /identity/v3/projects/2c8664e7ed9d45c6979d5730354fd429/users/c5cfcaa270604f7ea0b7a07a63968a7a/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:33.313679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.313972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.314250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.347535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] MFA Rules not processed for user `c5cfcaa270604f7ea0b7a07a63968a7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:33.349068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:33.369956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.416308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 943/1871] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:33.420718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.440493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965af61-3c1f-41fd-b870-dc54114d645e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.444812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.445404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.445683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.445951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.447897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:33.474951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:33.492613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd6eba-d02e-404a-af10-e41da2883917 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.499139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f2c06-44ff-4425-9f4b-7c48b358069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.499790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f2c06-44ff-4425-9f4b-7c48b358069a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.500235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f2c06-44ff-4425-9f4b-7c48b358069a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.500602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f2c06-44ff-4425-9f4b-7c48b358069a admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.501203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f2c06-44ff-4425-9f4b-7c48b358069a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.503247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f2c06-44ff-4425-9f4b-7c48b358069a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.519116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.521157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.531669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 929/1872] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:33.535940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 944/1873] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:33 2022] POST /identity/v3/projects => generated 389 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:33.540157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.546528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe53ec-c28d-4439-a4b1-7f798002beb3 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.546945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe53ec-c28d-4439-a4b1-7f798002beb3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.547592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe53ec-c28d-4439-a4b1-7f798002beb3 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.549299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 945/1874] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:56:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:56:33.567257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96771a1-3407-4965-9aed-923f4aeec8ce service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.602829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.603397 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.603793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.649904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:33.651728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:33.677569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.678273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.678626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.678987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.679813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.682178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.721594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 930/1875] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:33 2022] POST /identity/v3/users => generated 301 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:33.733074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.753040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cce8b34-ecca-46e6-ab1c-e764332e560a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.762463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.762463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.762463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.762839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.763472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.765839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.787931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 931/1876] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:33.789186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.800239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 946/1877] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:56:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:33.804545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6262955-501f-4cce-ac5c-30b0db5f75c8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.817555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.818960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:33.820183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.821063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.821367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.821669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] PATH_INFO: `/v3/projects/6dccc6bce9554cee91ad07477863c049/users/ccfbe4ddfc994a98bae637c80ec79a3c/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.837356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:33.840784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b23b89-5605-418d-81da-ea21c5abb1f2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:33.847682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dccc6bce9554cee91ad07477863c049, user_id=ccfbe4ddfc994a98bae637c80ec79a3c, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.847952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:33.848871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:33.849243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:33.849617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:33.854078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.862135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:33.868238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:33.877650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 932/1878] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:33 2022] PUT /identity/v3/projects/6dccc6bce9554cee91ad07477863c049/users/ccfbe4ddfc994a98bae637c80ec79a3c/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:33.889826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:33.897518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 947/1879] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:34.004874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea3220-d081-4080-b783-7514dde59629 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.010065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6930781c-c2af-4386-92bb-834e9ee1945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.010700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930781c-c2af-4386-92bb-834e9ee1945b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.011040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930781c-c2af-4386-92bb-834e9ee1945b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.011288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6930781c-c2af-4386-92bb-834e9ee1945b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.012054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930781c-c2af-4386-92bb-834e9ee1945b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.013732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6930781c-c2af-4386-92bb-834e9ee1945b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.031138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 933/1880] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:34.036068 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:34.047855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4225dbe-f9d1-4ad8-945e-df6b7b76222e service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.053431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.054194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.054547 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.054901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] PATH_INFO: `/v3/projects/6dccc6bce9554cee91ad07477863c049/users/ccfbe4ddfc994a98bae637c80ec79a3c/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.056979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dccc6bce9554cee91ad07477863c049, user_id=ccfbe4ddfc994a98bae637c80ec79a3c, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.059860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.071274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 948/1881] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:34 2022] PUT /identity/v3/projects/6dccc6bce9554cee91ad07477863c049/users/ccfbe4ddfc994a98bae637c80ec79a3c/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:34.082251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.082595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.082866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.116308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] MFA Rules not processed for user `ccfbe4ddfc994a98bae637c80ec79a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:34.117555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.205081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 934/1882] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:34.216022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:34.217922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.278207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e3e3c-898c-40dd-871a-02bb46929dc9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.282503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e37e4-70f6-48f1-9554-8c04df35285e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.283233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e37e4-70f6-48f1-9554-8c04df35285e service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.284164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e37e4-70f6-48f1-9554-8c04df35285e service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.284548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e37e4-70f6-48f1-9554-8c04df35285e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.295337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e37e4-70f6-48f1-9554-8c04df35285e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.297532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e37e4-70f6-48f1-9554-8c04df35285e service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.309292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 949/1883] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:34.474314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:34.537319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c4df4-c806-42fe-b112-1b1b9c9628b7 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.541828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.542452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.542736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.543008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] PATH_INFO: `/v3/users/c5cfcaa270604f7ea0b7a07a63968a7a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.544149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5cfcaa270604f7ea0b7a07a63968a7a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.546171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.559621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:34.596233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5cfcaa270604f7ea0b7a07a63968a7a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.605469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5cfcaa270604f7ea0b7a07a63968a7a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.606434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5cfcaa270604f7ea0b7a07a63968a7a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.620389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5cfcaa270604f7ea0b7a07a63968a7a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.621201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-bc3410e2-a3c9-41af-a4c7-3d4f57c32e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5cfcaa270604f7ea0b7a07a63968a7a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.630853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039540-742e-434a-b914-07bb49e9096c None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.631175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039540-742e-434a-b914-07bb49e9096c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.631797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa039540-742e-434a-b914-07bb49e9096c None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.633370 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 950/1884] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 07 15:56:34.635404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 935/1885] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:34 2022] DELETE /identity/v3/users/c5cfcaa270604f7ea0b7a07a63968a7a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:34.639809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.640100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.640379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.646448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.647139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.647802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.662718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:34.664055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.671454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:34.672764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.736330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:34.757572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 936/1886] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:34.758305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:34.759658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.760199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 951/1887] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:34.773560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:34.773890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe4eba-3f1d-41b6-b1d5-a4a76d63ad6e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.775101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.778693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.779328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.779636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.779912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] PATH_INFO: `/v3/users/bba0e831cc414c9c939fd334eda4f601` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.781149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bba0e831cc414c9c939fd334eda4f601)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.783229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.847768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f6fbc3-ced2-4af7-8854-da1139780f94 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.852476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bba0e831cc414c9c939fd334eda4f601'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.852910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.853637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.854036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.855082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.860806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bba0e831cc414c9c939fd334eda4f601'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.861449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bba0e831cc414c9c939fd334eda4f601'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.868074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.869673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.876012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bba0e831cc414c9c939fd334eda4f601'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.876807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bba0e831cc414c9c939fd334eda4f601'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:34.881517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 952/1888] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:34.882453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:34.883761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.886161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:34.897600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 937/1889] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:34 2022] DELETE /identity/v3/users/bba0e831cc414c9c939fd334eda4f601 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:34.901342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0ec47-8dfb-4d98-9702-19a392f5a366 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:34.970039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:34.970661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:34.970945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:34.971814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:34.974184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:34.987422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:34.989763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:34.999596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 953/1890] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:35.186983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:35.188600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:35.273357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6f899-39c9-4351-9349-f42555ff2e7d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:35.278529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:35.279590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:35.279746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:35.280084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] PATH_INFO: `/v3/projects/66f043feb09f4585af419e21a9566c54` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:35.281172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f043feb09f4585af419e21a9566c54)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:35.284686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:35.338882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:35.340444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:35.343452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f043feb09f4585af419e21a9566c54', 'request_id': 'req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.356846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e2f12a-93d3-4c7d-bee3-d2b74d7667ac service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:35.358637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f043feb09f4585af419e21a9566c54', 'request_id': 'req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.367446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f043feb09f4585af419e21a9566c54', 'request_id': 'req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.382907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] The token cache is being invalidate because project 66f043feb09f4585af419e21a9566c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:35.383229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.383992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.386051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 938/1891] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:35 2022] DELETE /identity/v3/projects/66f043feb09f4585af419e21a9566c54 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:35.390080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:35.392861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:35.446451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:35.447108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:35.447391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:35.447694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] PATH_INFO: `/v3/projects/2c8664e7ed9d45c6979d5730354fd429` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:35.449395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c8664e7ed9d45c6979d5730354fd429)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:35.452107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:35.490046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:35.496350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45c9a4-81fc-4dac-8868-d4411fc2f07a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:35.502217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:35.503616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:35.503890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:35.504190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] PATH_INFO: `/v3/users/1472c375a22e4323b7836b11b1e975aa` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:35.505507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1472c375a22e4323b7836b11b1e975aa)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:35.507647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:35.538532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c8664e7ed9d45c6979d5730354fd429', 'request_id': 'req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.550805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:35.562554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c8664e7ed9d45c6979d5730354fd429', 'request_id': 'req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.571716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c8664e7ed9d45c6979d5730354fd429', 'request_id': 'req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.583745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1472c375a22e4323b7836b11b1e975aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.584673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] The token cache is being invalidate because project 2c8664e7ed9d45c6979d5730354fd429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:35.584962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.585649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.591449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 954/1892] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:35 2022] DELETE /identity/v3/projects/2c8664e7ed9d45c6979d5730354fd429 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:35.592682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:35.593874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1472c375a22e4323b7836b11b1e975aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.594371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:35.594770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1472c375a22e4323b7836b11b1e975aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.617618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1472c375a22e4323b7836b11b1e975aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.619527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1472c375a22e4323b7836b11b1e975aa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.646854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 939/1893] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:35 2022] DELETE /identity/v3/users/1472c375a22e4323b7836b11b1e975aa => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:35.680300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d311f-9139-48ed-b9c5-e91ba2b466d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:35.750269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:35.751008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:35.751393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:35.751792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] PATH_INFO: `/v3/users/ccfbe4ddfc994a98bae637c80ec79a3c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:35.753461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccfbe4ddfc994a98bae637c80ec79a3c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:35.755947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:35.805952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccfbe4ddfc994a98bae637c80ec79a3c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.814715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccfbe4ddfc994a98bae637c80ec79a3c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.820273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:35.832342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccfbe4ddfc994a98bae637c80ec79a3c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.833022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:35.833477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccfbe4ddfc994a98bae637c80ec79a3c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.834700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-916641a1-1486-415e-8e6c-664f0139bd87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccfbe4ddfc994a98bae637c80ec79a3c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:35.834815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:35.852254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 955/1894] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:35 2022] DELETE /identity/v3/users/ccfbe4ddfc994a98bae637c80ec79a3c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:35.867268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1747a0f0-25af-4384-92ac-df4d916a0c27 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:35.941446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:35.942098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:35.942382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:35.942659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] PATH_INFO: `/v3/projects/636b87793ba84a8e9bd5c76833c5aa8f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:35.943856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=636b87793ba84a8e9bd5c76833c5aa8f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:35.945931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:35.998424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '636b87793ba84a8e9bd5c76833c5aa8f', 'request_id': 'req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.008618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '636b87793ba84a8e9bd5c76833c5aa8f', 'request_id': 'req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.015344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '636b87793ba84a8e9bd5c76833c5aa8f', 'request_id': 'req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.033171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] The token cache is being invalidate because project 636b87793ba84a8e9bd5c76833c5aa8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:36.034261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.035062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-485c8eb9-b7b5-4ae8-a940-ceefb6d0cb5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.039721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 940/1895] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:35 2022] DELETE /identity/v3/projects/636b87793ba84a8e9bd5c76833c5aa8f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:36.044309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.044650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.044912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.073296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:36.075671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.157852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 956/1896] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:36.166155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.166464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.166728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.176871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.178459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.194735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9978c3-27c3-494d-8151-de2dc0e79a17 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.195494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:36.196384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.203643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.204657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.205253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.206114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] PATH_INFO: `/v3/projects/6dccc6bce9554cee91ad07477863c049` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.207315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dccc6bce9554cee91ad07477863c049)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.209424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.212272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 941/1897] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:36.213243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.222831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509fa98-07ec-4991-859b-aa8a25e909f1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.228193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.228643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.228920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.229197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.229847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:36.230153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.231750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.244577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 942/1898] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:36.248906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.258825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7253b2ee-91df-4af1-8935-56b045607f10 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.268834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dccc6bce9554cee91ad07477863c049', 'request_id': 'req-5ac83c36-4151-48eb-9623-954b86ad074b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.272633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:36.277530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dccc6bce9554cee91ad07477863c049', 'request_id': 'req-5ac83c36-4151-48eb-9623-954b86ad074b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.289612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dccc6bce9554cee91ad07477863c049', 'request_id': 'req-5ac83c36-4151-48eb-9623-954b86ad074b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.304110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] The token cache is being invalidate because project 6dccc6bce9554cee91ad07477863c049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:36.304410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.305540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5ac83c36-4151-48eb-9623-954b86ad074b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:36.307627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 957/1899] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:36 2022] DELETE /identity/v3/projects/6dccc6bce9554cee91ad07477863c049 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:36.312403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781da1c-fad1-4a7e-aea8-ea100669e539 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.313269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781da1c-fad1-4a7e-aea8-ea100669e539 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.313704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781da1c-fad1-4a7e-aea8-ea100669e539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.322859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.323471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.323889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.324158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.324790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.326750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.334656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a781da1c-fad1-4a7e-aea8-ea100669e539 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:36.335987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a781da1c-fad1-4a7e-aea8-ea100669e539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.344990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 958/1900] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:36.356538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.356843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.357105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.362496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 943/1901] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/projects => generated 381 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:36.366702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.368668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.381617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:36.382893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.432670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:36.440507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1771b-ba92-4922-b112-687993719f79 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.445467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.446118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.446392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.446664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.447301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.448039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 959/1902] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:36.449257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.452205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.461402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0288ac9-3cd3-4f0e-a741-62ac61fe0538 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.466401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.467279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.467526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.467841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.468517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:36.468824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.470787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.476239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 944/1903] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/users => generated 291 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:36.477892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.487607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 960/1904] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:36.488465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968100ee-609b-4a42-aec7-1c7d40d1a08f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.488993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.493411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.494022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.494303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.494577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.495216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.496839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.499707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e58cf70-013a-49e0-9e2b-be8d9cb18be7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.504330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.504918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.505199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.505532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.506163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.510809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 945/1905] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:36.511032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.512484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.523690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df16510-ffa3-4efd-b55f-ede6a8aeca37 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.528643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.529237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.529517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.529792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.530463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.532130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.544190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 946/1906] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:36.546098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 961/1907] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:36.549029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.551122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.559680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba2015-b32c-4092-87f2-8571542168cb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.619847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396decd7-2977-483b-aa35-227788b63580 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.624710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc095c7-59a8-4f40-b9e1-05af79b50508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.625556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc095c7-59a8-4f40-b9e1-05af79b50508 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.625873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc095c7-59a8-4f40-b9e1-05af79b50508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.626239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc095c7-59a8-4f40-b9e1-05af79b50508 admin admin] PATH_INFO: `/v3/projects/c15ddf0292e34084a872d267a39bd446/users/2ce154aa384d4944a1f49db028974779/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.627866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.628781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.629167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.629496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.630440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.632710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.645642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc095c7-59a8-4f40-b9e1-05af79b50508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15ddf0292e34084a872d267a39bd446, user_id=2ce154aa384d4944a1f49db028974779, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.648577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc095c7-59a8-4f40-b9e1-05af79b50508 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.651504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:36.659989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 962/1908] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:36.664333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.667527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 947/1909] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:36 2022] PUT /identity/v3/projects/c15ddf0292e34084a872d267a39bd446/users/2ce154aa384d4944a1f49db028974779/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:36.675034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fb7cc-83c7-41f8-8025-11b5b5653489 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.679663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.680027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.680371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.717417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] MFA Rules not processed for user `2ce154aa384d4944a1f49db028974779`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:36.720159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.745015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.745698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.745981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.746256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.747385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.749073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.767303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 963/1910] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:36.768362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.779166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7fdff0-5654-481b-af2b-e49e0ead559f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.784765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.785394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.785786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.786076 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.786760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.788572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.801697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 964/1911] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:36.805954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.815907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2f17f-ef1f-4a17-b61c-e964c149c37b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.816355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 948/1912] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:36.820635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32a3f1-0efa-4248-a507-2a602c53ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.821010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.821443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32a3f1-0efa-4248-a507-2a602c53ff58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.821731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32a3f1-0efa-4248-a507-2a602c53ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.822010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32a3f1-0efa-4248-a507-2a602c53ff58 admin admin] PATH_INFO: `/v3/projects/b22b2ed00b4d4128997cad1a57782817/users/7e3986d178d641a68724a38a5581be9f/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.832148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c6416-d7c2-4335-8890-8d7130a1889d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.836711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.837328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.837606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.837800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32a3f1-0efa-4248-a507-2a602c53ff58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22b2ed00b4d4128997cad1a57782817, user_id=7e3986d178d641a68724a38a5581be9f, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.837923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.838795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.840612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32a3f1-0efa-4248-a507-2a602c53ff58 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.840722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.855405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 965/1913] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:36 2022] PUT /identity/v3/projects/b22b2ed00b4d4128997cad1a57782817/users/7e3986d178d641a68724a38a5581be9f/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:36.867072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.867343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.867645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.873717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 949/1914] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:36.874664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.896967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:36.909155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] MFA Rules not processed for user `7e3986d178d641a68724a38a5581be9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:36.911200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.945986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b92dd9-22b0-4137-a32b-cd55e9c6d49a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.951163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.951945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.952335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.952685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.953318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:36.955236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:36.977218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 950/1915] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:36.977535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 966/1916] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:36.981234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.988767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:36.990150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:36.991861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dc6559-9c57-4316-a678-98b7c13ccb30 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:36.996524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:36.997123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:36.997395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:36.997664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:36.998298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.000353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.010964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 951/1917] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:37.014836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.025550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42510749-b6e1-47e8-bac7-368a0496f549 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.030381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.031015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.031290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.031599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] PATH_INFO: `/v3/projects/bbe797bdf9964e89a9ce8f29aad7428a/users/2f14ac47d5cd4b98a8081fd1486f3311/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.046311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe797bdf9964e89a9ce8f29aad7428a, user_id=2f14ac47d5cd4b98a8081fd1486f3311, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.047261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ee812e-db93-4828-8caf-26ebf35af4fb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.049778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.051506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.052147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.052429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.052706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.062475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.063977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.064996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 952/1918] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:37 2022] PUT /identity/v3/projects/bbe797bdf9964e89a9ce8f29aad7428a/users/2f14ac47d5cd4b98a8081fd1486f3311/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:37.068979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.073520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 967/1919] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:36 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:37.094225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:37.095133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.107085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c9338-cb67-45a2-9913-68928c96b308 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.141265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bdb2-d246-4bb5-abf7-0e0868df2e90 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.147462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.148343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.148624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.149036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.150103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.152044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.175717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 953/1920] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:37 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:37.176750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.187443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e517ba-1dcc-4e76-b33e-75e2f1d0ba49 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.193196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d5f39-5c74-4d81-98a8-69adfe9f7179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.193932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5f39-5c74-4d81-98a8-69adfe9f7179 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.194265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.194422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5f39-5c74-4d81-98a8-69adfe9f7179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.194670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d5f39-5c74-4d81-98a8-69adfe9f7179 admin admin] PATH_INFO: `/v3/projects/bbe797bdf9964e89a9ce8f29aad7428a/users/2f14ac47d5cd4b98a8081fd1486f3311/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.195329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.195829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.196245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] PATH_INFO: `/v3/users/7e3986d178d641a68724a38a5581be9f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.196915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5f39-5c74-4d81-98a8-69adfe9f7179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbe797bdf9964e89a9ce8f29aad7428a, user_id=2f14ac47d5cd4b98a8081fd1486f3311, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.197681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3986d178d641a68724a38a5581be9f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.199767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d5f39-5c74-4d81-98a8-69adfe9f7179 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.199900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.211253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 954/1921] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:37 2022] PUT /identity/v3/projects/bbe797bdf9964e89a9ce8f29aad7428a/users/2f14ac47d5cd4b98a8081fd1486f3311/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:37.221862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.222153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.222430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.251259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:37.254107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3986d178d641a68724a38a5581be9f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.256132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] MFA Rules not processed for user `2f14ac47d5cd4b98a8081fd1486f3311`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:37.258252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:37.264230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3986d178d641a68724a38a5581be9f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.278303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3986d178d641a68724a38a5581be9f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.279463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3986d178d641a68724a38a5581be9f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.280027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6f77cb98-0817-4346-8851-2e875178f1f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3986d178d641a68724a38a5581be9f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.295621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 968/1922] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:37 2022] DELETE /identity/v3/users/7e3986d178d641a68724a38a5581be9f => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:37.299788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.300077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.300347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.324890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:37.326153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:37.349242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 955/1923] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:37.361276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.362699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:37.386329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 969/1924] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:37.396811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.398168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:37.422946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec92d22-5fd2-458a-b11c-2c53b978c42f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.463251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4f2f30-2e4f-4b7e-9c46-adf0ec5295e1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.467041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:37.467341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.468096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.468503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.468879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.481797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.483327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.484275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.484898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.485173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.485446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.499096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 970/1925] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:37.556116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.559104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.570694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 956/1926] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:37.669980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.680178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e160b3b-36a8-4714-8fef-822749d287e5 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.683821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.684426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.684696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.684981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.686816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:37.751629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.753660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687548f-c120-4892-986c-6c2eb8181180 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.762846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 971/1927] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:37.815351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:37.817624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:37.836796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb368cb-fd52-4ba7-ac1e-1706150de557 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:37.857003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:37.858406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.858690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.859094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] PATH_INFO: `/v3/projects/b22b2ed00b4d4128997cad1a57782817` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.860356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b22b2ed00b4d4128997cad1a57782817)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:37.864487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:37.927756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b22b2ed00b4d4128997cad1a57782817', 'request_id': 'req-b706b235-8094-4d0b-933a-615d60d55ef3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.939123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b22b2ed00b4d4128997cad1a57782817', 'request_id': 'req-b706b235-8094-4d0b-933a-615d60d55ef3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.949453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b22b2ed00b4d4128997cad1a57782817', 'request_id': 'req-b706b235-8094-4d0b-933a-615d60d55ef3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.963394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] The token cache is being invalidate because project b22b2ed00b4d4128997cad1a57782817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:37.963727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.964397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b706b235-8094-4d0b-933a-615d60d55ef3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:37.966388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 957/1928] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:37 2022] DELETE /identity/v3/projects/b22b2ed00b4d4128997cad1a57782817 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:37.971208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:37.971521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:37.971856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:37.982611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:37.998538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:38.011152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.086664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 972/1929] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:38.094656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.094949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.095252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.101872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.103287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.115131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79004a64-0dc5-43cc-a727-8a07994792eb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.119614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:38.119795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.120408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.120690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.120961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] PATH_INFO: `/v3/users/2ce154aa384d4944a1f49db028974779` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.121053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.122102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce154aa384d4944a1f49db028974779)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.124058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.130219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 958/1930] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:38.134549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.144520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ebd2-5d9a-4be3-98d4-d8c4dd114809 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.149167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.149753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.150040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.150308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.150948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:38.151252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.152868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.165280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 959/1931] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:38.169908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.180933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28b40a-dd6a-4032-9f27-d3a9de3fe813 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.185951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.186629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.186912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.187189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.187876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.190283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.191733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce154aa384d4944a1f49db028974779'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:38.200749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce154aa384d4944a1f49db028974779'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:38.217274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce154aa384d4944a1f49db028974779'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:38.218121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce154aa384d4944a1f49db028974779'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:38.219020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6fc4111-8fde-4c9b-9dc3-26b97ae0104d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce154aa384d4944a1f49db028974779'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:38.233641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 960/1932] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:38 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:38.233641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.233641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.234080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 973/1933] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:38 2022] DELETE /identity/v3/users/2ce154aa384d4944a1f49db028974779 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:38.239045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.239361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.239781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.260815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:38.262184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.269369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:38.305110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cbd7a-5941-4264-90eb-b803948a0344 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.332237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 974/1934] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:38.345515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.346938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.370896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.371426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.371773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.372297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.373290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:38.373464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.375776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.407628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 961/1935] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:38 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:38.409550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.420920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7b2c43-9e89-4261-9909-188b74b20d79 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.422536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7215cec4-58eb-4a26-b417-3b08b654a862 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.428244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.428854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.429136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.429410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.430059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.430786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.431412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.431683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.431781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.432100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.441667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.442838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 962/1936] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:38.443214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.446682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.454529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 975/1937] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:38.457770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f95998-c115-4850-80e8-327ee2d34e72 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.462486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c41a7-5015-4761-af36-6c569a794119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.463091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c41a7-5015-4761-af36-6c569a794119 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.463373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c41a7-5015-4761-af36-6c569a794119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.463694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c41a7-5015-4761-af36-6c569a794119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.464474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c41a7-5015-4761-af36-6c569a794119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.466110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c41a7-5015-4761-af36-6c569a794119 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.478611 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 963/1938] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:38.482865 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.495104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df378e7-a751-4a9f-bc1e-afcda6032cc6 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.499864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.500610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.500982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.501347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] PATH_INFO: `/v3/projects/d3be8f6fceb544be9ae495b5c2b8afeb/users/2ffe13a72e184882bfc026a5764e009b/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.517327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3be8f6fceb544be9ae495b5c2b8afeb, user_id=2ffe13a72e184882bfc026a5764e009b, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.520650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.536000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 976/1939] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:38 2022] PUT /identity/v3/projects/d3be8f6fceb544be9ae495b5c2b8afeb/users/2ffe13a72e184882bfc026a5764e009b/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:38.546923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.547440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.547753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.577986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] MFA Rules not processed for user `2ffe13a72e184882bfc026a5764e009b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:38.579223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.660342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 964/1940] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:38.671171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:38.672987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:38.736697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3f3a0-e8e1-4b4a-8dbd-94bf172e48a2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:38.740524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:38.741150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:38.741432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:38.741849 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:38.751879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:38.753688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:38.764019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 977/1941] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:39.109828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:39.114759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:39.201820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:39.232053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380c7e0-a401-411e-9756-83ec48f37cb1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:39.238628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:39.239269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:39.239567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:39.239860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] PATH_INFO: `/v3/projects/c15ddf0292e34084a872d267a39bd446` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:39.241023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15ddf0292e34084a872d267a39bd446)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:39.242973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:39.294112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15ddf0292e34084a872d267a39bd446', 'request_id': 'req-ef88577e-307e-40c9-bfea-429725fc8dab'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.301392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15ddf0292e34084a872d267a39bd446', 'request_id': 'req-ef88577e-307e-40c9-bfea-429725fc8dab'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.306985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15ddf0292e34084a872d267a39bd446', 'request_id': 'req-ef88577e-307e-40c9-bfea-429725fc8dab'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.318091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] The token cache is being invalidate because project c15ddf0292e34084a872d267a39bd446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:39.318383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.319038 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.320546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 965/1942] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:39 2022] DELETE /identity/v3/projects/c15ddf0292e34084a872d267a39bd446 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:39.324508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:39.335733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4319a4a-6544-4a52-ab91-283aedae050f service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:39.393617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:39.394330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:39.394628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:39.394927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] PATH_INFO: `/v3/users/2f14ac47d5cd4b98a8081fd1486f3311` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:39.396166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f14ac47d5cd4b98a8081fd1486f3311)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:39.398230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:39.438182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:39.446065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f14ac47d5cd4b98a8081fd1486f3311'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.453213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f14ac47d5cd4b98a8081fd1486f3311'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.465533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f14ac47d5cd4b98a8081fd1486f3311'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.466176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f14ac47d5cd4b98a8081fd1486f3311'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.466788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a579d72c-f0fe-4ee4-b0f1-88bde9a02ebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f14ac47d5cd4b98a8081fd1486f3311'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.479681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 978/1943] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:39 2022] DELETE /identity/v3/users/2f14ac47d5cd4b98a8081fd1486f3311 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:39.795585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:39.797026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:39.813876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88577e-307e-40c9-bfea-429725fc8dab admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:39.872087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:39.872865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:39.873795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:39.874077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] PATH_INFO: `/v3/projects/bbe797bdf9964e89a9ce8f29aad7428a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:39.875290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbe797bdf9964e89a9ce8f29aad7428a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:39.877943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:39.929978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbe797bdf9964e89a9ce8f29aad7428a', 'request_id': 'req-c747bbc6-ff19-4228-b639-3cbf09456e93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.939753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbe797bdf9964e89a9ce8f29aad7428a', 'request_id': 'req-c747bbc6-ff19-4228-b639-3cbf09456e93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.946600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbe797bdf9964e89a9ce8f29aad7428a', 'request_id': 'req-c747bbc6-ff19-4228-b639-3cbf09456e93'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.948768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:39.963752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] The token cache is being invalidate because project bbe797bdf9964e89a9ce8f29aad7428a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:39.964215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.965232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c747bbc6-ff19-4228-b639-3cbf09456e93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:39.966771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 966/1944] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:39 2022] DELETE /identity/v3/projects/bbe797bdf9964e89a9ce8f29aad7428a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:39.971765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:39.972106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:39.972383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:39.998140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:39.999879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:40.059419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 979/1945] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:40.067521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.067844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.068131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.087712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:40.088920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:40.097356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 967/1946] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:40.101748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.111396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d1ef1-b86c-497f-b4fb-32cd490c0ee2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.116448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.117042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.117320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.117594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.118257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:40.118557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.120192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.137395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 980/1947] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:40.142307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.151633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d5c9fa-df73-41d3-b312-c94932f568a9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.162024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.162732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.163019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.163297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.163982 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.166055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.199402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 968/1948] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:40 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:40.203685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.214882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc94be9-8c0c-4539-872d-e8532f81087f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.290318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.291228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.291562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.291837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.292488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.294884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.338546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 981/1949] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:40 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:40.342827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.356747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075148e-dcee-4e1d-b022-0c641a17d489 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.362124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.362807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.363163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.363573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.364296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.366134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.378048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 969/1950] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:40.382537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.393472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c66dfd4-a894-4b1d-a3b8-4b2381366378 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.398455 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2c6535-d609-40d5-862a-0df970081ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.399094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c6535-d609-40d5-862a-0df970081ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.399398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c6535-d609-40d5-862a-0df970081ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.399776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c6535-d609-40d5-862a-0df970081ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.400492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c6535-d609-40d5-862a-0df970081ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.402173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2c6535-d609-40d5-862a-0df970081ed8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.414349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 982/1951] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:40.419691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.433763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb95426-740e-42e7-a8e9-fb28dd794021 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.439316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.440710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.441110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.441453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] PATH_INFO: `/v3/projects/81967df28c234cd5866009e5f2cc7803/users/8e41ab6e922d4434a6da82a91f0a6166/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.460295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81967df28c234cd5866009e5f2cc7803, user_id=8e41ab6e922d4434a6da82a91f0a6166, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.464034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.477788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 970/1952] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:40 2022] PUT /identity/v3/projects/81967df28c234cd5866009e5f2cc7803/users/8e41ab6e922d4434a6da82a91f0a6166/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:40.490444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.491463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.491931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.521928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] MFA Rules not processed for user `8e41ab6e922d4434a6da82a91f0a6166`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:40.523239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:40.534584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:40.598337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 983/1953] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:40.603160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.681011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccd7c9-6dbf-45b9-973b-03c9f228c947 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.685763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.686369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.686662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.686941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.687597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.689493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.719567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 971/1954] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:40 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:40.723753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.734836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68698779-94a6-4aff-b0e5-157ed7666d61 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.803096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.804012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.804343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.804846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.805730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.808884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.837360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 984/1955] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:40 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:40.841468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.857515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed2014-b058-475b-835b-d589b34cf47a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.864894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.865871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.866364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.866764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.867849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.869693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.887477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 972/1956] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:40.891843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.902990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9979e8c-8a70-4511-ba25-9959adb92660 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:40.907914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:40.909090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:40.909368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:40.909639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] PATH_INFO: `/v3/projects/5083347398b14c3a950dc0f55f2888c5/users/10bbc88ee4ab474b9980fcf1ef8d9e9b/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:40.932655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5083347398b14c3a950dc0f55f2888c5, user_id=10bbc88ee4ab474b9980fcf1ef8d9e9b, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:40.932655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:40.942704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 985/1957] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:40 2022] PUT /identity/v3/projects/5083347398b14c3a950dc0f55f2888c5/users/10bbc88ee4ab474b9980fcf1ef8d9e9b/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:40.946587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:40.957377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beab8ea1-0a74-41cd-827b-f7631486a2af admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:41.013316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03174913-2da5-40f6-b659-0fc3ae92f84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:41.014067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174913-2da5-40f6-b659-0fc3ae92f84c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:41.014941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174913-2da5-40f6-b659-0fc3ae92f84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:41.015256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03174913-2da5-40f6-b659-0fc3ae92f84c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:41.016002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03174913-2da5-40f6-b659-0fc3ae92f84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:41.017626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03174913-2da5-40f6-b659-0fc3ae92f84c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:41.028808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 973/1958] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:41.032913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:41.047179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee82a17-3da6-4379-9a8d-8b704dbaf502 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:41.052718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:41.053487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:41.053898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:41.054178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] PATH_INFO: `/v3/projects/5083347398b14c3a950dc0f55f2888c5/users/10bbc88ee4ab474b9980fcf1ef8d9e9b/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:41.056162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5083347398b14c3a950dc0f55f2888c5, user_id=10bbc88ee4ab474b9980fcf1ef8d9e9b, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:41.058777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:41.069192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 986/1959] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:41 2022] PUT /identity/v3/projects/5083347398b14c3a950dc0f55f2888c5/users/10bbc88ee4ab474b9980fcf1ef8d9e9b/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:41.079334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:41.079790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:41.080099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:41.087240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:41.109937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] MFA Rules not processed for user `10bbc88ee4ab474b9980fcf1ef8d9e9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:41.111127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:41.178568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 974/1960] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:41.189411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:41.191303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:41.248877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc22d24-43b8-444e-b765-cc69b082f983 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:41.252602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1fefc1-bd3f-487c-8fed-6dfc59adb0ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:41.253263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fefc1-bd3f-487c-8fed-6dfc59adb0ad service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:41.253838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fefc1-bd3f-487c-8fed-6dfc59adb0ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:41.253838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1fefc1-bd3f-487c-8fed-6dfc59adb0ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:41.271671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:41.313778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fefc1-bd3f-487c-8fed-6dfc59adb0ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:41.314442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1fefc1-bd3f-487c-8fed-6dfc59adb0ad service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:41.324340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 987/1961] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:47.407619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:47.409679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.493887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7242a7dd-2162-4a1f-99f1-f4f621c433d9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:47.502101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:47.504541 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.504799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.505442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] PATH_INFO: `/v3/users/1831430fc92f447d931ed32f83872f48` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.506700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1831430fc92f447d931ed32f83872f48)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:47.519499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:47.569767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1831430fc92f447d931ed32f83872f48'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.580976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1831430fc92f447d931ed32f83872f48'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.582089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1831430fc92f447d931ed32f83872f48'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.602033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1831430fc92f447d931ed32f83872f48'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.603816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1831430fc92f447d931ed32f83872f48'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.620710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 975/1962] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:47 2022] DELETE /identity/v3/users/1831430fc92f447d931ed32f83872f48 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:47.625408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.626090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.626090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.648059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:47.649295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.651677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:47.653183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.665844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:47.723957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 988/1963] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:47.734315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:47.735769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.735882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10edc826-3c44-4bfc-98f3-522f05eb3b8a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:47.740718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:47.741309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.741585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.741856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] PATH_INFO: `/v3/users/5d595d4be1284a63b4d393bfae951853` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.743029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d595d4be1284a63b4d393bfae951853)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:47.745013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:47.801464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d595d4be1284a63b4d393bfae951853'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.811252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d595d4be1284a63b4d393bfae951853'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.812012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d595d4be1284a63b4d393bfae951853'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.815354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d92737-4365-4ca5-b1cb-c8413b68e997 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:47.819416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e83fc-afe7-496f-837f-2af6aa8f3e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:47.820369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e83fc-afe7-496f-837f-2af6aa8f3e25 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.820479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e83fc-afe7-496f-837f-2af6aa8f3e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.820713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e83fc-afe7-496f-837f-2af6aa8f3e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.826260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d595d4be1284a63b4d393bfae951853'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.827124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-82833c57-33e5-4b35-8a7d-9c017441e916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d595d4be1284a63b4d393bfae951853'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:47.833850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e83fc-afe7-496f-837f-2af6aa8f3e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:47.836355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e83fc-afe7-496f-837f-2af6aa8f3e25 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:47.841280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 976/1964] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:47 2022] DELETE /identity/v3/users/5d595d4be1284a63b4d393bfae951853 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:47.850710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.851000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.851526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.865772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 989/1965] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:47.868203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9cc30-4dad-4f2e-b7b7-2dc680d14131 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.868560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9cc30-4dad-4f2e-b7b7-2dc680d14131 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.868828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c9cc30-4dad-4f2e-b7b7-2dc680d14131 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.870231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 990/1966] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 15:56:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:56:47.873332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:47.874777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.879401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:47.879910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:47.880194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:47.910765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:47.912119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.956060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 977/1967] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:47.968993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:47.971073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:47.997418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:48.023816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 991/1968] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 15:56:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:48.026400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:48.039258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:48.041548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:48.065209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f713ff5-acb5-467e-813f-c3fdc30ad0bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:48.075107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbb4f0-f99d-4f0f-9e88-ccd0447e48c9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:48.079464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:48.080670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:48.080925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:48.081195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:48.085136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:48.085899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:48.086338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:48.086500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:48.098070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:48.099476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:48.099742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:48.101955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:48.112788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 992/1969] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:48.120192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 978/1970] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:48.220438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:48.221980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:48.237024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a031f3-7c5a-4011-bcae-6c9a86e9a8ad service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:48.242441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:48.243706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:48.243942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:48.244211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] PATH_INFO: `/v3/projects/06fface65be44d029bf953b65d2ab7f7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:48.245358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06fface65be44d029bf953b65d2ab7f7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:48.247290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:48.302930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06fface65be44d029bf953b65d2ab7f7', 'request_id': 'req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.311322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06fface65be44d029bf953b65d2ab7f7', 'request_id': 'req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.318580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06fface65be44d029bf953b65d2ab7f7', 'request_id': 'req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.329460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] The token cache is being invalidate because project 06fface65be44d029bf953b65d2ab7f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:48.329744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.330459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.332038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 993/1971] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:48 2022] DELETE /identity/v3/projects/06fface65be44d029bf953b65d2ab7f7 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:48.336014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:48.346634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae7d33a-9463-4741-800b-90e93f22af12 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:48.412885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:48.413527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:48.413929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:48.414266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] PATH_INFO: `/v3/users/bad3996187684897a4f5d89823006511` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:48.415532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad3996187684897a4f5d89823006511)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:48.417537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:48.419120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:48.420737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:48.434874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40a5d3-c0a6-4f0e-b33e-7cc15c900496 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:48.442014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:48.442671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:48.442959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:48.443238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] PATH_INFO: `/v3/projects/3a9fa2b25ee449c5a5a3206766a4bec2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:48.444655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a9fa2b25ee449c5a5a3206766a4bec2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:48.446510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:48.476966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad3996187684897a4f5d89823006511'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.489694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad3996187684897a4f5d89823006511'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.490648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad3996187684897a4f5d89823006511'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.499425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a9fa2b25ee449c5a5a3206766a4bec2', 'request_id': 'req-b4772380-5e95-4553-bde8-f8e1f2ba27b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.505177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad3996187684897a4f5d89823006511'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.505856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad3996187684897a4f5d89823006511'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.507531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a9fa2b25ee449c5a5a3206766a4bec2', 'request_id': 'req-b4772380-5e95-4553-bde8-f8e1f2ba27b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.513799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a9fa2b25ee449c5a5a3206766a4bec2', 'request_id': 'req-b4772380-5e95-4553-bde8-f8e1f2ba27b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.518655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 979/1972] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:48 2022] DELETE /identity/v3/users/bad3996187684897a4f5d89823006511 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:48.529199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] The token cache is being invalidate because project 3a9fa2b25ee449c5a5a3206766a4bec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:48.529504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.530255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.534815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 994/1973] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:48 2022] DELETE /identity/v3/projects/3a9fa2b25ee449c5a5a3206766a4bec2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:48.536018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:48.537469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:48.590152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:48.607826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08dfef-cb0d-461e-96ce-1c1c9a708b8a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:48.613665 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:48.614305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:48.615370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:48.615875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] PATH_INFO: `/v3/users/23027aaf70a1428f872f895a425e9915` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:48.617037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23027aaf70a1428f872f895a425e9915)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:48.619002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:48.684974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23027aaf70a1428f872f895a425e9915'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.693574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23027aaf70a1428f872f895a425e9915'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.694359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23027aaf70a1428f872f895a425e9915'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.709724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23027aaf70a1428f872f895a425e9915'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.712486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23027aaf70a1428f872f895a425e9915'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:48.733432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 980/1974] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:48 2022] DELETE /identity/v3/users/23027aaf70a1428f872f895a425e9915 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:48.911896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:48.914266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:48.973137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:48.991832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4772380-5e95-4553-bde8-f8e1f2ba27b7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.055188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.055881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.056220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.056484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] PATH_INFO: `/v3/projects/6e856e39d29a456881fae740b904d2ef` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.057589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e856e39d29a456881fae740b904d2ef)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.059853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.081391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.082914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.112161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be6a13-728c-496d-b4db-6fc1fca34da0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.117278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.118037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.118405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.118765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] PATH_INFO: `/v3/projects/b4ed96f51c414eee9abb40f068ff71c4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.120050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ed96f51c414eee9abb40f068ff71c4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.122237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.138242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e856e39d29a456881fae740b904d2ef', 'request_id': 'req-16a37f93-83bd-43e4-819f-e5e7aa154f45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.148506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e856e39d29a456881fae740b904d2ef', 'request_id': 'req-16a37f93-83bd-43e4-819f-e5e7aa154f45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.159648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e856e39d29a456881fae740b904d2ef', 'request_id': 'req-16a37f93-83bd-43e4-819f-e5e7aa154f45'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.169295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] The token cache is being invalidate because project 6e856e39d29a456881fae740b904d2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:49.175342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.175342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-16a37f93-83bd-43e4-819f-e5e7aa154f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.175342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 995/1975] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:48 2022] DELETE /identity/v3/projects/6e856e39d29a456881fae740b904d2ef => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:49.176995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5ee0e-4616-4a1c-b5fd-8f8511b07755 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.177350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5ee0e-4616-4a1c-b5fd-8f8511b07755 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.177694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb5ee0e-4616-4a1c-b5fd-8f8511b07755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.190332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ed96f51c414eee9abb40f068ff71c4', 'request_id': 'req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.198524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4fb5ee0e-4616-4a1c-b5fd-8f8511b07755 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:49.199921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4fb5ee0e-4616-4a1c-b5fd-8f8511b07755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.200869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ed96f51c414eee9abb40f068ff71c4', 'request_id': 'req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.207519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ed96f51c414eee9abb40f068ff71c4', 'request_id': 'req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.218951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] The token cache is being invalidate because project b4ed96f51c414eee9abb40f068ff71c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:49.219239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.219908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-69ae6c6e-dd6d-48e6-80a6-01d54a3742b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:49.222297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 981/1976] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:49 2022] DELETE /identity/v3/projects/b4ed96f51c414eee9abb40f068ff71c4 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:49.227246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59baf-92dc-4691-b486-fa9cab1abe1c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.227679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59baf-92dc-4691-b486-fa9cab1abe1c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.228105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf59baf-92dc-4691-b486-fa9cab1abe1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.248495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0cf59baf-92dc-4691-b486-fa9cab1abe1c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:49.249722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0cf59baf-92dc-4691-b486-fa9cab1abe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.275380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 996/1977] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:49.283574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.283882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.284158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.302157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:49.304473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:49.305657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.313839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 997/1978] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:49.317942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.318959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 982/1979] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:49.326367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.326646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.326918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.327296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189cf62-5639-4a13-b4c8-d7312467f6e1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.332251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.332864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.333133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.333440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.334213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:49.334550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.336329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.346629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:49.347874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.348799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 998/1980] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:49.353209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.359387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 983/1981] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:49.361320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:49.364601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.364848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d9085-b118-4b3f-a51b-9e79da43e682 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.371227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.371879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.372169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.372441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.373085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.374948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ef379-8c6b-4784-9ede-c6c885a4ea2e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.375564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.380601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.381193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.381468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.381733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.382376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:49.382678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.384280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.401325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 984/1982] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:56:49.402403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.403873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 999/1983] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:49.407817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.414222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89459a0-2b3f-4e74-8a2e-822367228c58 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.472805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.473419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.473700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.473976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.474789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.477020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.481991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485e107-7c08-42e2-8a86-8425b2287f41 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.486683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.487291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.487655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.487934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.488565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.490534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.504678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 985/1984] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:49.508575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.516049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1000/1985] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:49.517272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.527749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb6c0e-6bb8-4e8d-8b79-2470d5c88f1e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.572897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6b0dd-d717-456c-b6a6-97855d2946d3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.577795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.578427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.578715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.579018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.579680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.582128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.586316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.586933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.587216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.587490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.588170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.589754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.599808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1001/1986] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:49.604040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.606240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 986/1987] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/users => generated 283 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:49.610040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.614483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cdd324-416b-4ae9-8fbf-b6c9eb2418d2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.619624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.620286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.620520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.620996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e48d7-a80a-4321-9d62-8f45bf72a13d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.621166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.621421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.623062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.625627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.626329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.626621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.626952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.627638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.629337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.634022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1002/1988] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:49.638319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.642932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 987/1989] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:49.643921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.649238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78ed53-f852-4d21-9888-21e90091da7a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.653367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae93274-0a68-4c9b-bba3-68504d81cb34 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.654471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464fd65-83f7-4bf4-8704-a9733696a7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.655119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fd65-83f7-4bf4-8704-a9733696a7b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.655397 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fd65-83f7-4bf4-8704-a9733696a7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.655712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464fd65-83f7-4bf4-8704-a9733696a7b8 admin admin] PATH_INFO: `/v3/projects/3f20afaccd17462a923d94c92cfcc13c/users/4fb9bee43d574f65a62188380a77a077/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.657896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.658552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.658832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.659108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.659767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.661342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.670675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464fd65-83f7-4bf4-8704-a9733696a7b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f20afaccd17462a923d94c92cfcc13c, user_id=4fb9bee43d574f65a62188380a77a077, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.673028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 988/1990] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:49.675627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464fd65-83f7-4bf4-8704-a9733696a7b8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.676587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.687707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26fc48a-3d44-4a4a-95a0-09855d5a47e8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.688062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1003/1991] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:49 2022] PUT /identity/v3/projects/3f20afaccd17462a923d94c92cfcc13c/users/4fb9bee43d574f65a62188380a77a077/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:49.699251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.699718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.700249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.709846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:49.735090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] MFA Rules not processed for user `4fb9bee43d574f65a62188380a77a077`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:49.736868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.757947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:49.758875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2f003-ca6f-488f-80d2-ec6b29548ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.759675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2f003-ca6f-488f-80d2-ec6b29548ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.759934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2f003-ca6f-488f-80d2-ec6b29548ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.760201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2f003-ca6f-488f-80d2-ec6b29548ef1 admin admin] PATH_INFO: `/v3/projects/1f3cdceb8e2c4c058e932947124668d3/users/c4cc41b3c95945399197c14b1455d9b4/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.780097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2f003-ca6f-488f-80d2-ec6b29548ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f3cdceb8e2c4c058e932947124668d3, user_id=c4cc41b3c95945399197c14b1455d9b4, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:49.782691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2f003-ca6f-488f-80d2-ec6b29548ef1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:49.797760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 989/1992] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:49 2022] PUT /identity/v3/projects/1f3cdceb8e2c4c058e932947124668d3/users/c4cc41b3c95945399197c14b1455d9b4/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:49.809500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1004/1993] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:49.809708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.810160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.810460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:49.824509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.826406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.848463 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] MFA Rules not processed for user `c4cc41b3c95945399197c14b1455d9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:49.850073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:49.906696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9381cae-ef8a-4343-be83-5a1eaac3e630 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:49.924399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 990/1994] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:49.929630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:49.965756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:49.966351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:49.966639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:49.966938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.011617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb3050-e21b-4cd9-bf01-d6d225ee7372 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.016754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.017449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.018272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.018550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.019183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.021185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.034938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.036840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.047476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1005/1995] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:50.055398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 991/1996] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:49 2022] POST /identity/v3/projects => generated 367 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:50.056675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.067014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.068182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0df211-afa7-4a3c-bbd2-30ceb8de6fd1 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.102499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:50.102675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:50.132024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.132659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.132939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.133209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.133856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.135812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.141215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f157dc4-5535-43ae-9b35-972f8f7b328f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.145950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.146668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.147002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.147342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] PATH_INFO: `/v3/users/4fb9bee43d574f65a62188380a77a077` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.148517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fb9bee43d574f65a62188380a77a077)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.150428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.168451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1006/1997] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:50 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:50.172829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.186801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd592b6-e6ca-42d8-8b85-e38703caeed8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.193321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.193923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.194200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.194612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.195356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.197119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.205666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fb9bee43d574f65a62188380a77a077'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:50.216572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1007/1998] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:50.217810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fb9bee43d574f65a62188380a77a077'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:50.218782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fb9bee43d574f65a62188380a77a077'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:50.221589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.231985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bd2c5c-db2a-415b-b6eb-21fc62c50598 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.235210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fb9bee43d574f65a62188380a77a077'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:50.236067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-1f6d1edb-6f80-4214-b1d7-24e4753a0ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fb9bee43d574f65a62188380a77a077'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:50.238193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.239049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.239330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.239638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] PATH_INFO: `/v3/projects/ad1cd1a6cc73410989c8d3414e6f1944/users/89a9f7ea660049f682d7dc75b56ac9f2/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.251331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 992/1999] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:50 2022] DELETE /identity/v3/users/4fb9bee43d574f65a62188380a77a077 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:50.256006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.256136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.256337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.259088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1cd1a6cc73410989c8d3414e6f1944, user_id=89a9f7ea660049f682d7dc75b56ac9f2, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.261786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.278999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1008/2000] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:50 2022] PUT /identity/v3/projects/ad1cd1a6cc73410989c8d3414e6f1944/users/89a9f7ea660049f682d7dc75b56ac9f2/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:50.280080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:50.281320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.283732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.285495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.356435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 993/2001] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:50.369226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.372224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.380271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7c23a5-2627-4e16-8fc9-dababde7165f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.387564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.388285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.388647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.388979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.389698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.391379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.408678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1009/2002] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:50.409860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.420284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d61915-f11e-4014-9258-19d758a964b0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.428716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e8e30-5a70-433c-9c73-ac68b06c06f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.429750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e8e30-5a70-433c-9c73-ac68b06c06f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.430070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e8e30-5a70-433c-9c73-ac68b06c06f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.430438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e8e30-5a70-433c-9c73-ac68b06c06f3 admin admin] PATH_INFO: `/v3/projects/ad1cd1a6cc73410989c8d3414e6f1944/users/89a9f7ea660049f682d7dc75b56ac9f2/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.433561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e8e30-5a70-433c-9c73-ac68b06c06f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad1cd1a6cc73410989c8d3414e6f1944, user_id=89a9f7ea660049f682d7dc75b56ac9f2, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.436943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e8e30-5a70-433c-9c73-ac68b06c06f3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.437916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:50.449001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1010/2003] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:50 2022] PUT /identity/v3/projects/ad1cd1a6cc73410989c8d3414e6f1944/users/89a9f7ea660049f682d7dc75b56ac9f2/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:50.454655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aba2f8-e93e-4f31-b788-180bb22823d1 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.459941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.460787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.460787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.497941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] MFA Rules not processed for user `89a9f7ea660049f682d7dc75b56ac9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:50.500194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.539213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.539865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.540210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.540883 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.570667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:50.581423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1011/2004] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:50.592875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.594236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.603036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.604495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.614635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 994/2005] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:50.653880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9b94-7cd1-47ea-9304-1c42b390cdb2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.657474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.658728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.659004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.659275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.670333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.672202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.683625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1012/2006] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:50.902098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.913114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7deec2-7b51-4ca3-8dcf-767703beeb65 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.919021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.919021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.919021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.919021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.920062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:50.920239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.921503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:50.947183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fd48a-0f78-4213-9cc3-abaa03dde115 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:50.952393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:50.953041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:50.953324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:50.953599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] PATH_INFO: `/v3/projects/3f20afaccd17462a923d94c92cfcc13c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:50.956279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f20afaccd17462a923d94c92cfcc13c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.957475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:50.997047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:50.998507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeadb28-90cb-4880-957d-afd55bf3573b service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.008964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 995/2007] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:51.012853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f20afaccd17462a923d94c92cfcc13c', 'request_id': 'req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:51.021428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f20afaccd17462a923d94c92cfcc13c', 'request_id': 'req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:51.029046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f20afaccd17462a923d94c92cfcc13c', 'request_id': 'req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:51.040608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] The token cache is being invalidate because project 3f20afaccd17462a923d94c92cfcc13c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:51.040841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:51.041504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-ab6a2ee6-e842-4d45-ad89-f298cb8affeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:51.043642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1013/2008] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:50 2022] DELETE /identity/v3/projects/3f20afaccd17462a923d94c92cfcc13c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:51.048210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.048503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.049310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.069411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:51.070903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:51.136268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:51.164386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 996/2009] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:51.172548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.173128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.174125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.201285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:51.202639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:51.214831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1014/2010] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:51.219261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.229439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff62fd-16fb-4cd5-a75c-b36d640b4a9e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.234152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.234767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.235046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.235483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.236728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:51.237035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.238678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.251413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 997/2011] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:51.255837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.266189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4280877e-873a-4180-95b6-16d3f3c732b0 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.271962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.272619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.272906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.273185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.274050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.276114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.321410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1015/2012] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:51.328707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.340120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ebadd1-1bb3-4907-8b9c-3904b64066d5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.412817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.413610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.413894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.414167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.414830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.416980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.440094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 998/2013] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:51.444302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.455849 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff6940-7a49-4efe-9090-fd78e053ff08 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.461344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.461977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.462263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.462536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.463189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.464853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.476505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1016/2014] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:51.480413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.491361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e687f3c-3f49-4c10-9ac3-29d1d04b8ad1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.496271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186f1b9-f1c2-45b5-a8c8-8c8677bb938c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.496882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f1b9-f1c2-45b5-a8c8-8c8677bb938c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.497166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f1b9-f1c2-45b5-a8c8-8c8677bb938c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.497468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186f1b9-f1c2-45b5-a8c8-8c8677bb938c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.498121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f1b9-f1c2-45b5-a8c8-8c8677bb938c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.499806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186f1b9-f1c2-45b5-a8c8-8c8677bb938c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.523915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 999/2015] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:51.524016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.531379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa49230-af11-4396-96f2-2250e7c2b2b8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.536539 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.537241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.537682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.537960 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] PATH_INFO: `/v3/projects/a1c6b6cc203041eb9946dba956ca8410/users/3bd5c9af160d467b8c79c5b30b198824/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.555052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1c6b6cc203041eb9946dba956ca8410, user_id=3bd5c9af160d467b8c79c5b30b198824, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.557683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.573435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1017/2016] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:51 2022] PUT /identity/v3/projects/a1c6b6cc203041eb9946dba956ca8410/users/3bd5c9af160d467b8c79c5b30b198824/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:51.584201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.584394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.584645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.619024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] MFA Rules not processed for user `3bd5c9af160d467b8c79c5b30b198824`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:51.620515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:51.693589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1000/2017] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:51.698128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.717561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:51.762381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae6c440-7801-459f-a387-b0b9c62e0006 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.767183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.768336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.768622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.768898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.769545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.771522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.798727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1018/2018] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/projects => generated 383 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:51.802822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.813589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fd349-7b46-4ac1-b2e0-1e2944c9e757 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.853128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:51.870781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.871384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.871707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.871988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.872627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.874550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.896290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1001/2019] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:51 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:51.900232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.910874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be267dd-4198-4319-8065-b40768f34952 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.915470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.916104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.916396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.916675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.917315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.918902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.930503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1019/2020] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:51.933798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.944526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae951b-f61a-4e25-be3e-bf1175074f2a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.949100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e871c0-3d32-417e-8ffd-2d617f4cacf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.949696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e871c0-3d32-417e-8ffd-2d617f4cacf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.949977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e871c0-3d32-417e-8ffd-2d617f4cacf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.950255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e871c0-3d32-417e-8ffd-2d617f4cacf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.950896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e871c0-3d32-417e-8ffd-2d617f4cacf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:51.952531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e871c0-3d32-417e-8ffd-2d617f4cacf8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:51.963344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1002/2021] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:51.967310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:51.978123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce2a88-12af-4351-9bee-46d5e2d381fd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:51.983032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:51.983725 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:51.984012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:51.984284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] PATH_INFO: `/v3/projects/802ad102691343d780aede8036ea52a4/users/e5743a4806f54e8496121df40704d81e/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:51.999174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802ad102691343d780aede8036ea52a4, user_id=e5743a4806f54e8496121df40704d81e, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.002410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.014713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1020/2022] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:51 2022] PUT /identity/v3/projects/802ad102691343d780aede8036ea52a4/users/e5743a4806f54e8496121df40704d81e/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:52.026149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.026446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.026727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.054751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] MFA Rules not processed for user `e5743a4806f54e8496121df40704d81e`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:52.055992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:52.117440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1003/2023] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:52.121780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.193523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdaf07f-3666-4c60-9f9a-9a6bcdf39213 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.198351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.198961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.199273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.199518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.200335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.202291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.232915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1021/2024] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:52 2022] POST /identity/v3/projects => generated 383 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:52.235064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.248284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16607216-379b-4c4f-9348-e6ecaea37fca None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.308509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.309129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.309412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.309762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.310302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.312296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.342357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1004/2025] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:52 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:52.343582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.354902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb13cdc-65e8-474f-b582-c54ddb35cd90 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.360556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.361198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.361482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.361755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.362427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.364075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.379925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1022/2026] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:52.381094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.391614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aba2fdf-4a3c-4b8f-8247-59285b5519d9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.396355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.397020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.397305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.397588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] PATH_INFO: `/v3/projects/460a22db6e3d4c4b937e2f0ec1a2c3a3/users/648927f6c93a4b91848426467eded08b/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.415080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460a22db6e3d4c4b937e2f0ec1a2c3a3, user_id=648927f6c93a4b91848426467eded08b, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.417795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.431983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1005/2027] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:52 2022] PUT /identity/v3/projects/460a22db6e3d4c4b937e2f0ec1a2c3a3/users/648927f6c93a4b91848426467eded08b/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:52.435921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.445887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedf109-a2d4-46fc-8348-fdf37889e1cc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.488710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:52.508751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2bb3af-d591-49d5-88bf-850d76622d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.509521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bb3af-d591-49d5-88bf-850d76622d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.509858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bb3af-d591-49d5-88bf-850d76622d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.510139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2bb3af-d591-49d5-88bf-850d76622d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.510780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bb3af-d591-49d5-88bf-850d76622d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.512420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2bb3af-d591-49d5-88bf-850d76622d52 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.523178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1023/2028] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:52.527423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.539410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8db9ff-7fe4-4716-9778-e13a31fe74f2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.544480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.545144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.545424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.545700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] PATH_INFO: `/v3/projects/460a22db6e3d4c4b937e2f0ec1a2c3a3/users/648927f6c93a4b91848426467eded08b/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.547651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=460a22db6e3d4c4b937e2f0ec1a2c3a3, user_id=648927f6c93a4b91848426467eded08b, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.550577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.561127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1006/2029] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:52 2022] PUT /identity/v3/projects/460a22db6e3d4c4b937e2f0ec1a2c3a3/users/648927f6c93a4b91848426467eded08b/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:52.571451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.571816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.572105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.609922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] MFA Rules not processed for user `648927f6c93a4b91848426467eded08b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:52.611334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:52.695251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1024/2030] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:52 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:56:52.709653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:52.712600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:52.719034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:52.786629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c03295-9075-45b3-b2f5-d9b046e86d81 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:52.790363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:52.791068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:52.791320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:52.791644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:52.864173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:52.865621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:52.876007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1007/2031] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:55.547814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:55.558328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3c8b6-1d84-4f53-aaf5-ab6b59d8cd61 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:55.562090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a592049-85b7-48a3-b6be-eb6d093f9e5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:55.562774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a592049-85b7-48a3-b6be-eb6d093f9e5b service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:55.563117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a592049-85b7-48a3-b6be-eb6d093f9e5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:55.563430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a592049-85b7-48a3-b6be-eb6d093f9e5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:55.623788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a592049-85b7-48a3-b6be-eb6d093f9e5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:55.625605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a592049-85b7-48a3-b6be-eb6d093f9e5b service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:55.636301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1025/2032] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:56.011253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:56.013027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.100893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a2b575-8556-4238-a3da-f8581ec40599 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:56.106597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:56.107318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:56.107826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:56.120100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] PATH_INFO: `/v3/users/2ffe13a72e184882bfc026a5764e009b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:56.125042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ffe13a72e184882bfc026a5764e009b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:56.129406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:56.197169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ffe13a72e184882bfc026a5764e009b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:56.207141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ffe13a72e184882bfc026a5764e009b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:56.208347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ffe13a72e184882bfc026a5764e009b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:56.214415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:56.227827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ffe13a72e184882bfc026a5764e009b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:56.228575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ffe13a72e184882bfc026a5764e009b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:56.245086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1008/2033] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:56 2022] DELETE /identity/v3/users/2ffe13a72e184882bfc026a5764e009b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:56.250006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:56.250508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:56.251028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:56.292785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:56.313473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.329093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:56.332009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.390736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1026/2034] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:56.390736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:56.390736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.397876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f5874c-a0a1-4949-aa9e-3d67f3e08fc5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:56.398416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:56.401879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:56.402474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:56.402756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:56.403193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:56.405118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.407217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7d39b-2792-44e7-9c8c-a697c6155ab6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:56.411672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:56.411876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:56.412230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:56.412593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:56.414530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.477835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:56.479372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:56.488636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:56.489046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1009/2035] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:56.489977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:56.490107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:56.491362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:56.500223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1027/2036] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:56.559450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f59853c-8441-430e-811e-a0b46fdc135f service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:56.565274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3ad9b-f8b7-4e5f-b5ca-03c7303ff15f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:56.566156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3ad9b-f8b7-4e5f-b5ca-03c7303ff15f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:56.566449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3ad9b-f8b7-4e5f-b5ca-03c7303ff15f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:56.566756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3ad9b-f8b7-4e5f-b5ca-03c7303ff15f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:56.583335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3ad9b-f8b7-4e5f-b5ca-03c7303ff15f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:56.583335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3ad9b-f8b7-4e5f-b5ca-03c7303ff15f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:56.607991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1010/2037] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:56:57.087141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.088680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:57.110463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d18c0-bf27-48f8-976b-cb7432b05aaa service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.116285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.127686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.127686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.127686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] PATH_INFO: `/v3/projects/d3be8f6fceb544be9ae495b5c2b8afeb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.134940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3be8f6fceb544be9ae495b5c2b8afeb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.138556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.209915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3be8f6fceb544be9ae495b5c2b8afeb', 'request_id': 'req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:57.219360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3be8f6fceb544be9ae495b5c2b8afeb', 'request_id': 'req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:57.228509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3be8f6fceb544be9ae495b5c2b8afeb', 'request_id': 'req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:57.242431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] The token cache is being invalidate because project d3be8f6fceb544be9ae495b5c2b8afeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:56:57.243004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:57.243796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3f4296fa-a9b6-4f8d-85fb-87d887fcfc2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:57.245646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1028/2038] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:56:57 2022] DELETE /identity/v3/projects/d3be8f6fceb544be9ae495b5c2b8afeb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:57.250214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.251680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.252467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.275085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:57.276365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:57.307157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:57.369058 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1011/2039] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:57.377230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.377525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.377807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.398062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:57.399286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:57.407344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1029/2040] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:57.411688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.421992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f799a-4056-4f56-94ef-81efe05c5441 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.426847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.427436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.427736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.428005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.428647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:56:57.428953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.430534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.443579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1012/2041] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:56:57.447198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.456504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c46d3-bbf8-4ebf-baf5-e0a5b6f6d63e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.461269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.461871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.462153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.462576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.463060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.465019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.498484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1030/2042] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:57 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:57.500361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.511904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ebd56b-b425-42ba-ac55-66d5880d1f7f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.581885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.582859 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.583705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.584066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.584800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.587021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.612204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1013/2043] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:57 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:56:57.616117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.630687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24287d-df9e-4324-bb6c-284da2abc7a9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.635731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.636510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.639123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.645370 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.646062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.649101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.662735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1031/2044] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:57.666716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.679423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9352600-1623-4f49-bec8-c23bb8d78fcb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.684888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721bf09-91b2-4c33-a0b7-26f0b049e6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.685560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721bf09-91b2-4c33-a0b7-26f0b049e6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.685864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721bf09-91b2-4c33-a0b7-26f0b049e6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.686182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721bf09-91b2-4c33-a0b7-26f0b049e6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.686851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721bf09-91b2-4c33-a0b7-26f0b049e6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.689663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721bf09-91b2-4c33-a0b7-26f0b049e6b6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.701664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1014/2045] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:57.706568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.718899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfed188-8df6-4501-ad62-b98643671ed6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:57.727071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:57.727781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.728073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.728406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] PATH_INFO: `/v3/projects/94f50f8251964f149a94e25397797e6a/users/15957a08541d4ee993dd3b4632fde187/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.746549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f50f8251964f149a94e25397797e6a, user_id=15957a08541d4ee993dd3b4632fde187, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:57.749362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:57.765021 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1032/2046] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:57 2022] PUT /identity/v3/projects/94f50f8251964f149a94e25397797e6a/users/15957a08541d4ee993dd3b4632fde187/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:57.780057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:57.780057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:57.780057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:57.806534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] MFA Rules not processed for user `15957a08541d4ee993dd3b4632fde187`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:57.807988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:57.884709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1015/2047] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:57.889253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:57.908236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:58.002220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5ae60-907e-4e81-8e3c-6b5e75d5a5b2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:58.013494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:58.014237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:58.014619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:58.015420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:58.016194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:58.018263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:58.125178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1033/2048] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:57 2022] POST /identity/v3/projects => generated 383 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:58.130517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:58.183194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d13b38-039f-4918-9cd2-fd04420c5185 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:58.250561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:58.345389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:58.346200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:58.348163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:58.348960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:58.349866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:58.354083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:58.390001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1016/2049] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:58 2022] POST /identity/v3/users => generated 292 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:58.394317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:58.416161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862ca59-4178-45ed-a99d-6ef3df6692f6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:58.437661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:58.438490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:58.439840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:58.440530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:58.443077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:58.444848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:58.474023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1034/2050] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:58.475068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:58.489104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b788ff-3864-4313-81ec-75be69088b26 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:58.496204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a17471c-49d8-4333-97b2-6bb154e93cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:58.498972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17471c-49d8-4333-97b2-6bb154e93cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:58.499372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17471c-49d8-4333-97b2-6bb154e93cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:58.500171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a17471c-49d8-4333-97b2-6bb154e93cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:58.501091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17471c-49d8-4333-97b2-6bb154e93cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:58.503598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a17471c-49d8-4333-97b2-6bb154e93cfc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:58.521318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1017/2051] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:58.526272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:58.542521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88311ad0-2df1-4c65-8134-0117c18965ef admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:58.558927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:58.561194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:58.562805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:58.563188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] PATH_INFO: `/v3/projects/ed157ffb648c4a109a241b0f37bcc1fd/users/9767feb1a012487dbae3b1bc9505b011/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:58.610468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed157ffb648c4a109a241b0f37bcc1fd, user_id=9767feb1a012487dbae3b1bc9505b011, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:58.618136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:58.648908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1035/2052] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:58 2022] PUT /identity/v3/projects/ed157ffb648c4a109a241b0f37bcc1fd/users/9767feb1a012487dbae3b1bc9505b011/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:58.660275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:58.660674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:58.661052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:58.720425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] MFA Rules not processed for user `9767feb1a012487dbae3b1bc9505b011`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:58.727761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:58.993775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1018/2053] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:59.006561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.153053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a800bfd-ae7e-4564-864a-576dbd0d32b7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.194532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.195773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.195773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.195773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.196580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.199970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.256630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1036/2054] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:56:59 2022] POST /identity/v3/projects => generated 385 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:59.260988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.285201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.288734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8185927-84b7-4486-a43a-ed3cd961dff0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.307101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:59.387287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.387414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.387718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.387963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.389053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.391029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.416581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1019/2055] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:56:59 2022] POST /identity/v3/users => generated 299 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:56:59.422150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.431764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:59.435350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08420c73-10e3-48c3-aa19-532f516d7ea5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.450049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853d837-5419-46d0-9300-b44c6705df24 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.455325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.456139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.456426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.456700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] PATH_INFO: `/v3/users/c4cc41b3c95945399197c14b1455d9b4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.458434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4cc41b3c95945399197c14b1455d9b4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.460511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.469033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.470107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.471279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.475848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.478032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.479954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.491822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1020/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:56:59.496181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.507938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e460cdc-2ca3-44d6-8cb2-ec6d4b664f5e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.512861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.513511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.513795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.514095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] PATH_INFO: `/v3/projects/2380162a06bc4cbc8ed96a38ac4d94be/users/47bae3ce49b943d984a2a65b29c7ead6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.523284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4cc41b3c95945399197c14b1455d9b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:59.525635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:56:59.533258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4cc41b3c95945399197c14b1455d9b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:59.534851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2380162a06bc4cbc8ed96a38ac4d94be, user_id=47bae3ce49b943d984a2a65b29c7ead6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.539049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.552766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4cc41b3c95945399197c14b1455d9b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:59.553534 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4cc41b3c95945399197c14b1455d9b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:59.557036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-df9186e5-6928-44ec-a77a-7d5b6f19f621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4cc41b3c95945399197c14b1455d9b4'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:56:59.557176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1021/2057] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:59 2022] PUT /identity/v3/projects/2380162a06bc4cbc8ed96a38ac4d94be/users/47bae3ce49b943d984a2a65b29c7ead6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:56:59.558086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.559959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:59.569299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1037/2058] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:56:59 2022] DELETE /identity/v3/users/c4cc41b3c95945399197c14b1455d9b4 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:59.573590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.574507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.574794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.603594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:59.604597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:59.668576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6440a-5f82-4e96-812a-5e457e8ec754 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.701414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1038/2059] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:56:59.717855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.720791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:59.760942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.761670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.761835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.762115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.762949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.764578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.777344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1022/2060] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:56:59.783656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:56:59.807857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959e296-cebf-4d7f-bfac-008a110353b0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.820034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76876ef9-0c8d-4b90-a8c8-a5802b400e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.820651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876ef9-0c8d-4b90-a8c8-a5802b400e06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.820934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876ef9-0c8d-4b90-a8c8-a5802b400e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.821210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76876ef9-0c8d-4b90-a8c8-a5802b400e06 admin admin] PATH_INFO: `/v3/projects/2380162a06bc4cbc8ed96a38ac4d94be/users/47bae3ce49b943d984a2a65b29c7ead6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.823436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76876ef9-0c8d-4b90-a8c8-a5802b400e06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2380162a06bc4cbc8ed96a38ac4d94be, user_id=47bae3ce49b943d984a2a65b29c7ead6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.826694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76876ef9-0c8d-4b90-a8c8-a5802b400e06 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.827165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0008435-65ee-47a7-b7d3-7a28a1a4eff3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:56:59.837424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:56:59.837943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.839789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.840070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.849183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1023/2061] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:56:59 2022] PUT /identity/v3/projects/2380162a06bc4cbc8ed96a38ac4d94be/users/47bae3ce49b943d984a2a65b29c7ead6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:56:59.862536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:56:59.863178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:56:59.863686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:56:59.864586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:56:59.866662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:56:59.889189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1039/2062] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:56:59.900337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] MFA Rules not processed for user `47bae3ce49b943d984a2a65b29c7ead6`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:56:59.901766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:56:59.989261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1024/2063] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:56:59 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:00.003706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:00.006922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:00.039741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:00.193007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea4fd5-96c8-4f2f-bd6d-7905444563de service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:00.280844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:00.281569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:00.281920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:00.282264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:00.285452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:00.338880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:00.341005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:00.353691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:00.384826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:00.387515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:00.404704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1040/2064] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:56:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:00.440810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb78a17-4c81-4a15-964d-ad1a375a9111 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:00.445627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:00.446171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:00.446445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:00.446729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] PATH_INFO: `/v3/projects/1f3cdceb8e2c4c058e932947124668d3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:00.448010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f3cdceb8e2c4c058e932947124668d3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:00.450104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:00.525251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f3cdceb8e2c4c058e932947124668d3', 'request_id': 'req-7e92c37c-a409-4806-88bb-0526eea0f79b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.534107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f3cdceb8e2c4c058e932947124668d3', 'request_id': 'req-7e92c37c-a409-4806-88bb-0526eea0f79b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.543389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f3cdceb8e2c4c058e932947124668d3', 'request_id': 'req-7e92c37c-a409-4806-88bb-0526eea0f79b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.567802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] The token cache is being invalidate because project 1f3cdceb8e2c4c058e932947124668d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:00.568098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.568824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.570828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1025/2065] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:00 2022] DELETE /identity/v3/projects/1f3cdceb8e2c4c058e932947124668d3 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:00.574812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:00.586169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d22eac2-c039-48f6-bc6e-8559f18b41c8 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:00.686650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:00.688324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:00.690912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:00.691940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:00.692601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:00.693098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] PATH_INFO: `/v3/users/89a9f7ea660049f682d7dc75b56ac9f2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:00.694495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a9f7ea660049f682d7dc75b56ac9f2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:00.696765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:00.709297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92c37c-a409-4806-88bb-0526eea0f79b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:00.716970 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:00.717885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:00.718420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:00.719471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] PATH_INFO: `/v3/users/3bd5c9af160d467b8c79c5b30b198824` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:00.720332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd5c9af160d467b8c79c5b30b198824)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:00.724242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:00.734363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:00.781651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a9f7ea660049f682d7dc75b56ac9f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.791045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a9f7ea660049f682d7dc75b56ac9f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.806978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a9f7ea660049f682d7dc75b56ac9f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.807753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a9f7ea660049f682d7dc75b56ac9f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.808466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8ab4a7ba-51ce-4d29-9d47-800475e3e754 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a9f7ea660049f682d7dc75b56ac9f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.813443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd5c9af160d467b8c79c5b30b198824'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.825325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd5c9af160d467b8c79c5b30b198824'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.826081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd5c9af160d467b8c79c5b30b198824'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.833838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1041/2066] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:00 2022] DELETE /identity/v3/users/89a9f7ea660049f682d7dc75b56ac9f2 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:00.843715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd5c9af160d467b8c79c5b30b198824'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.843715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd5c9af160d467b8c79c5b30b198824'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:00.863577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1026/2067] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:00 2022] DELETE /identity/v3/users/3bd5c9af160d467b8c79c5b30b198824 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:00.866782 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:00.867192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:00.867475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:00.894205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:00.895442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:00.982635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1042/2068] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:00.991740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:00.993382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.001132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:01.061760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8350a1d-96fe-45ae-ab25-0a7921aa0b7d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.073416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6c793-ef9c-408b-8dd2-3df9ed7b6fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.074189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6c793-ef9c-408b-8dd2-3df9ed7b6fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.075356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6c793-ef9c-408b-8dd2-3df9ed7b6fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.075752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6c793-ef9c-408b-8dd2-3df9ed7b6fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.088518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6c793-ef9c-408b-8dd2-3df9ed7b6fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.090482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6c793-ef9c-408b-8dd2-3df9ed7b6fd1 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.101731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1027/2069] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:57:01.277698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.279254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.291434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243cb93-d245-4ce1-9e3f-970586ecdf45 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.296761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.297616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.297986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.298184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] PATH_INFO: `/v3/projects/ad1cd1a6cc73410989c8d3414e6f1944` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.299405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad1cd1a6cc73410989c8d3414e6f1944)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.301363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.365642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad1cd1a6cc73410989c8d3414e6f1944', 'request_id': 'req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.377184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad1cd1a6cc73410989c8d3414e6f1944', 'request_id': 'req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.385199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad1cd1a6cc73410989c8d3414e6f1944', 'request_id': 'req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.396750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] The token cache is being invalidate because project ad1cd1a6cc73410989c8d3414e6f1944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:01.397039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.397821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-cdeb20f0-ae9e-41e6-bbbd-69583d46881e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.399319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1043/2070] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:01 2022] DELETE /identity/v3/projects/ad1cd1a6cc73410989c8d3414e6f1944 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:01.405945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.405945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.405945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.425176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:01.426474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.486593 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1028/2071] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:01.494427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.495048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.495672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.525484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:01.526627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.539202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1044/2072] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:01.540439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.552197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1b4067-070e-46a5-855e-c07600a2df6b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.566966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.567752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.568084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.568459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.569482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:01.570394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.572430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.583574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.585422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.596715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1029/2073] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:01.597847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.607810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1f5f0-4395-4851-9de8-3167ab9620a1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.619992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.620775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.621113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.621328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b4067-7be4-43ff-89ba-4d285a6a640b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.621433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] PATH_INFO: `/v3/projects/a1c6b6cc203041eb9946dba956ca8410` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.623348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1c6b6cc203041eb9946dba956ca8410)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.625736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.629259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.629879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.630216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.630498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.631142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.633337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.648015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:01.666439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1030/2074] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:01 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:01.670290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.689003 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1c6b6cc203041eb9946dba956ca8410', 'request_id': 'req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.696667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1c6b6cc203041eb9946dba956ca8410', 'request_id': 'req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.703084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1c6b6cc203041eb9946dba956ca8410', 'request_id': 'req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.714157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] The token cache is being invalidate because project a1c6b6cc203041eb9946dba956ca8410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:01.714543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.715638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.717224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1045/2075] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:01 2022] DELETE /identity/v3/projects/a1c6b6cc203041eb9946dba956ca8410 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:01.721501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.723365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.739537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8026b23-38f4-4977-82ec-e999ffec5cbc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.756606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:01.795930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e949ba11-44b1-4619-9db2-ceb51a0ef0dd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.799330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.799988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.800269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.800543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.801152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.801277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.801948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.802295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.802636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] PATH_INFO: `/v3/users/e5743a4806f54e8496121df40704d81e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.803198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.803952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5743a4806f54e8496121df40704d81e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.806033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.823938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1031/2076] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:01 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:01.827879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.830147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.843740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f996dd-08a1-4019-aed5-4d92733cb8eb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.849136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.849752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.850009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.850288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.850964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.852875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.860434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5743a4806f54e8496121df40704d81e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.863431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1032/2077] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:57:01.867486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.868597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5743a4806f54e8496121df40704d81e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.879375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ac58f1-43b4-48d7-a2eb-dd8a0322161d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.883862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5743a4806f54e8496121df40704d81e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.884699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5743a4806f54e8496121df40704d81e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.884897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.885477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4a2a1986-86de-41ab-92da-584bc065a59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5743a4806f54e8496121df40704d81e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:01.885586 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.885749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.886015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:01.894588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:01.894588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:01.904450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1033/2078] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:01.908254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1046/2079] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:01 2022] DELETE /identity/v3/users/e5743a4806f54e8496121df40704d81e => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:01.911955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:01.913679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:01.927116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7335cd9-49b0-4906-8c73-e6c2645af56e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:01.986604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:01.987362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:01.987673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:01.988022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] PATH_INFO: `/v3/projects/8f934e9bb8f74b798e5e43f8a1975f5a/users/3dfe18ccb6634691b719e692d131f4b7/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:02.006168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f934e9bb8f74b798e5e43f8a1975f5a, user_id=3dfe18ccb6634691b719e692d131f4b7, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:02.009941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:02.043192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1034/2080] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:01 2022] PUT /identity/v3/projects/8f934e9bb8f74b798e5e43f8a1975f5a/users/3dfe18ccb6634691b719e692d131f4b7/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:02.047811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:02.047811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:02.047811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:02.077800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] MFA Rules not processed for user `3dfe18ccb6634691b719e692d131f4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:02.079143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:02.088812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:02.151240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1047/2081] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:02.162942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:02.164763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:02.193633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:02.195412 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:02.254431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751fbf50-b137-4425-8e99-d7e45cb51e9c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:02.258140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:02.258738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:02.259013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:02.259287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:02.264985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:02.270016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:02.271527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:02.274024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c20763-d8bd-4714-ac1a-84c8faf69539 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:02.280159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:02.281114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:02.281402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:02.281658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] PATH_INFO: `/v3/projects/802ad102691343d780aede8036ea52a4` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:02.283293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=802ad102691343d780aede8036ea52a4)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:02.285244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:02.285954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1035/2082] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:02.338702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802ad102691343d780aede8036ea52a4', 'request_id': 'req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.348158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802ad102691343d780aede8036ea52a4', 'request_id': 'req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.355136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802ad102691343d780aede8036ea52a4', 'request_id': 'req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.371155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] The token cache is being invalidate because project 802ad102691343d780aede8036ea52a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:02.371522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.372201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.373982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1048/2083] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:02 2022] DELETE /identity/v3/projects/802ad102691343d780aede8036ea52a4 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:02.378126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:02.394345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffb4853-df57-460d-afba-25b9b94a896c service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:02.464428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:02.465063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:02.465343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:02.465616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] PATH_INFO: `/v3/users/648927f6c93a4b91848426467eded08b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:02.466781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] RBAC: Authorizing `identity:delete_user(user_id=648927f6c93a4b91848426467eded08b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:02.468746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:02.519461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '648927f6c93a4b91848426467eded08b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.531130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '648927f6c93a4b91848426467eded08b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.531818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '648927f6c93a4b91848426467eded08b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.547133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '648927f6c93a4b91848426467eded08b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.547862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5771f14d-1065-4f8a-8796-cd16d5050648 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '648927f6c93a4b91848426467eded08b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:02.560764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1036/2084] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:02 2022] DELETE /identity/v3/users/648927f6c93a4b91848426467eded08b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:02.953006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:02.955432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:02.970233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f20c50-183d-4e8a-8c6b-92b6f499dd4f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.012684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:03.030162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.030797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.031078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.031350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] PATH_INFO: `/v3/projects/460a22db6e3d4c4b937e2f0ec1a2c3a3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.032514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] RBAC: Authorizing `identity:delete_project(project_id=460a22db6e3d4c4b937e2f0ec1a2c3a3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.034429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.080969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '460a22db6e3d4c4b937e2f0ec1a2c3a3', 'request_id': 'req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:03.088867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '460a22db6e3d4c4b937e2f0ec1a2c3a3', 'request_id': 'req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:03.095625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '460a22db6e3d4c4b937e2f0ec1a2c3a3', 'request_id': 'req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:03.106264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] The token cache is being invalidate because project 460a22db6e3d4c4b937e2f0ec1a2c3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:03.106552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:03.107192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a27ced4e-fc56-4e92-a9a5-151a7bb7d070 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:03.108829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1049/2085] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:02 2022] DELETE /identity/v3/projects/460a22db6e3d4c4b937e2f0ec1a2c3a3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:03.115072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.115394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.115727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.135755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:03.137300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:03.195890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1037/2086] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:03.203859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.204177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.204998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.225207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:03.226673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:03.234569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1050/2087] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:03.238707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.248389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852aa0f-3c34-4aec-b1da-022173e79b07 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.253108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.253706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.254073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.254347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.255024 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:03.255324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.256950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.265454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:03.269462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1038/2088] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:03.273810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.283097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802174a3-6b16-4b70-9eb6-d7a3a4ae59e2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.287806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.288433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.288717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.288992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.289621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.291524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.318523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1051/2089] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:03 2022] POST /identity/v3/projects => generated 389 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:03.322554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.332196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7417821-f127-4640-96e5-11fbb5883ac2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.388273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.388918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.389202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.389478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.390113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.392076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.412818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1039/2090] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:03 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:03.416843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.427042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9df01-858b-44de-b8ea-4a25181970d5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.431657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.432262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.432547 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.432820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.433487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.435086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.445407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1052/2091] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:03.449236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.459530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3fc8f3-4f4c-45a0-b155-6f7f9b482ff6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.464429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b3c2f-d52a-4b5a-8a9f-54d0ee352e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.465035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3c2f-d52a-4b5a-8a9f-54d0ee352e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.465589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3c2f-d52a-4b5a-8a9f-54d0ee352e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.465589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b3c2f-d52a-4b5a-8a9f-54d0ee352e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.466265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b3c2f-d52a-4b5a-8a9f-54d0ee352e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.467954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b3c2f-d52a-4b5a-8a9f-54d0ee352e63 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.479318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1040/2092] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:03.483425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.493361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe59ea-e00c-4e13-9898-866bcabb4181 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.497942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.498595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.498884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.499162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] PATH_INFO: `/v3/projects/4a0f587828dc4ae6980ed70a20291381/users/d36682108c7040bfa841ea0dd1f634be/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.514878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a0f587828dc4ae6980ed70a20291381, user_id=d36682108c7040bfa841ea0dd1f634be, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.518571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.533039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1053/2093] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:03 2022] PUT /identity/v3/projects/4a0f587828dc4ae6980ed70a20291381/users/d36682108c7040bfa841ea0dd1f634be/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:03.544301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.544592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.544910 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.575448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] MFA Rules not processed for user `d36682108c7040bfa841ea0dd1f634be`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:03.576647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:03.646840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1041/2094] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:03.658436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:03.660191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:03.722505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925efe5-f8ea-4fe8-ac0f-8a36c6f8356b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:03.726430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:03.727047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:03.727328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:03.727658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:03.736294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:03.737767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:03.746343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1054/2095] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:06.310417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:06.311871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:06.324112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3baeb2-8c0c-43bb-8d06-9cbb88e04f1e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:06.327871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:06.328478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:06.328763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:06.329102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:06.330835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:06.390845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:06.392363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:06.401555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1042/2096] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:57:07.054065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:07.067590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d6682-0a83-4e2f-a71e-57ba668b84e8 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:07.070189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:07.070913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:07.071423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:07.071758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:07.073785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:07.084214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:07.148210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:07.150202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174d22db-c1ba-4467-92f4-2b484f37c784 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:07.165262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1055/2097] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:07.323824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:07.326015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:07.337134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:07.426831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f89e84-ec79-4e73-a6a8-d48b2476f2be service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:07.433064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:07.433858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:07.434239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:07.434616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] PATH_INFO: `/v3/users/8e41ab6e922d4434a6da82a91f0a6166` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:07.435964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e41ab6e922d4434a6da82a91f0a6166)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:07.438059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:07.500386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e41ab6e922d4434a6da82a91f0a6166'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:07.508447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e41ab6e922d4434a6da82a91f0a6166'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:07.509418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e41ab6e922d4434a6da82a91f0a6166'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:07.521406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e41ab6e922d4434a6da82a91f0a6166'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:07.522025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e41ab6e922d4434a6da82a91f0a6166'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:07.534501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1043/2098] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:07 2022] DELETE /identity/v3/users/8e41ab6e922d4434a6da82a91f0a6166 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:07.541268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:07.541409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:07.541911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:07.566341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:07.567974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:07.641628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1056/2099] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:07.651490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:07.652919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:07.744471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af60682e-4d7f-4a22-9171-ea63df91433a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:07.749741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:07.750345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:07.751009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:07.751009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:07.761365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:07.762860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:07.772996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1044/2100] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:08.057913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:08.060108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:08.090876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7ab0cd-4efb-4979-b0d2-e21820750b1a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:08.098025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:08.098845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:08.099332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:08.099810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] PATH_INFO: `/v3/projects/81967df28c234cd5866009e5f2cc7803` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:08.101372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] RBAC: Authorizing `identity:delete_project(project_id=81967df28c234cd5866009e5f2cc7803)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:08.103309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:08.159074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81967df28c234cd5866009e5f2cc7803', 'request_id': 'req-dce08526-28d7-4009-9789-fc6e756d8ffc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.167821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81967df28c234cd5866009e5f2cc7803', 'request_id': 'req-dce08526-28d7-4009-9789-fc6e756d8ffc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.180229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81967df28c234cd5866009e5f2cc7803', 'request_id': 'req-dce08526-28d7-4009-9789-fc6e756d8ffc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.198058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] The token cache is being invalidate because project 81967df28c234cd5866009e5f2cc7803 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:08.198585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.199332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.201135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1057/2101] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:08 2022] DELETE /identity/v3/projects/81967df28c234cd5866009e5f2cc7803 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:08.205100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:08.217343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18362d-c250-45e7-b684-dd8e4cabda1e service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:08.248298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:08.292867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:08.293411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:08.293854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:08.294342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] PATH_INFO: `/v3/users/10bbc88ee4ab474b9980fcf1ef8d9e9b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:08.295666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bbc88ee4ab474b9980fcf1ef8d9e9b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:08.300113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:08.377773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bbc88ee4ab474b9980fcf1ef8d9e9b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.388607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bbc88ee4ab474b9980fcf1ef8d9e9b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.389371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bbc88ee4ab474b9980fcf1ef8d9e9b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.406760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bbc88ee4ab474b9980fcf1ef8d9e9b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.407601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c21c1877-d264-4216-bf3d-447833e1ff8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bbc88ee4ab474b9980fcf1ef8d9e9b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.424353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1045/2102] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:08 2022] DELETE /identity/v3/users/10bbc88ee4ab474b9980fcf1ef8d9e9b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:08.784949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:08.786414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:08.809989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce08526-28d7-4009-9789-fc6e756d8ffc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:08.840803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:08.875178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:08.875956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:08.876341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:08.876680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] PATH_INFO: `/v3/projects/5083347398b14c3a950dc0f55f2888c5` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:08.877909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5083347398b14c3a950dc0f55f2888c5)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:08.879968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:08.929513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5083347398b14c3a950dc0f55f2888c5', 'request_id': 'req-78da1b56-9262-46d6-813b-fded32b3c6f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.936664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5083347398b14c3a950dc0f55f2888c5', 'request_id': 'req-78da1b56-9262-46d6-813b-fded32b3c6f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.943887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5083347398b14c3a950dc0f55f2888c5', 'request_id': 'req-78da1b56-9262-46d6-813b-fded32b3c6f2'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.954936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] The token cache is being invalidate because project 5083347398b14c3a950dc0f55f2888c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:08.955228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.955910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-78da1b56-9262-46d6-813b-fded32b3c6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:08.957408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1058/2103] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:08 2022] DELETE /identity/v3/projects/5083347398b14c3a950dc0f55f2888c5 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:08.963342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:08.963655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:08.963925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:08.985255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:08.986723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:09.057657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1046/2104] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:09.060555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.061017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.061109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.082465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:09.083726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:09.094727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1059/2105] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:09.096496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.107203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3243567-ce6a-4347-9622-d57c0cc2c3c4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.114102 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.114710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.115120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.115403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.116087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:09.116395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.117994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.131370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1047/2106] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:09.136090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.150463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d0b566-daa6-46f9-aa84-96c1d6416f97 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.159413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.159413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.159413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.159413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.159848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.159848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.198234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1060/2107] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:09 2022] POST /identity/v3/projects => generated 359 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:09.202156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.216577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc158ca8-e8d4-47ed-8d00-55b1b4b6bddc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.254927 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:09.293148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.293913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.294196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.294476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.295688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.297518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.323746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1048/2108] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:09 2022] POST /identity/v3/users => generated 280 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:09.325253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.342923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ae689-49d8-4d33-b247-22b6b487899b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.352315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.353156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.353443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.353721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.354379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.356046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.367623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1061/2109] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:09.371810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.382501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be54e71-87e7-407e-9283-f38cafdf1109 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.388900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62249407-4416-4273-8f11-8e08cf14ca79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.389678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62249407-4416-4273-8f11-8e08cf14ca79 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.390129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62249407-4416-4273-8f11-8e08cf14ca79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.390561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62249407-4416-4273-8f11-8e08cf14ca79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.391372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62249407-4416-4273-8f11-8e08cf14ca79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.394343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62249407-4416-4273-8f11-8e08cf14ca79 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.406143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1049/2110] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:09.410422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.422078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79123b70-f387-4b44-9122-7c49c19d6223 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.426574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.427378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.427706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.427982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] PATH_INFO: `/v3/projects/e37fda2e939149409275a3ad0053a262/users/308361e94fbc47d099f2922994c5c0dc/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.444641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37fda2e939149409275a3ad0053a262, user_id=308361e94fbc47d099f2922994c5c0dc, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.447298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.469429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1062/2111] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:09 2022] PUT /identity/v3/projects/e37fda2e939149409275a3ad0053a262/users/308361e94fbc47d099f2922994c5c0dc/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:09.481624 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.482192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.482474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.512417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] MFA Rules not processed for user `308361e94fbc47d099f2922994c5c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:09.513807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:09.584209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1050/2112] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:09.624447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.629753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:09.698256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce32e54-0d45-496d-ac43-1a16612e73c6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.704256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44e39f-3b6f-468d-9fd8-e75b9d21796d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.705396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e39f-3b6f-468d-9fd8-e75b9d21796d service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.705681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e39f-3b6f-468d-9fd8-e75b9d21796d service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.705957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44e39f-3b6f-468d-9fd8-e75b9d21796d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.714793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44e39f-3b6f-468d-9fd8-e75b9d21796d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.716238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44e39f-3b6f-468d-9fd8-e75b9d21796d service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.719932 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:09.728868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1063/2113] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:09.748355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:09.817925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e8a6b9-d1b0-4392-9a05-53d150f8e82b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:09.824301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:09.824953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.825218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.825491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] PATH_INFO: `/v3/users/308361e94fbc47d099f2922994c5c0dc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.826631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] RBAC: Authorizing `identity:delete_user(user_id=308361e94fbc47d099f2922994c5c0dc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:09.828600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:09.879142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:09.880948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '308361e94fbc47d099f2922994c5c0dc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:09.890664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '308361e94fbc47d099f2922994c5c0dc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:09.891571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '308361e94fbc47d099f2922994c5c0dc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:09.904395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '308361e94fbc47d099f2922994c5c0dc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:09.905054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '308361e94fbc47d099f2922994c5c0dc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:09.917190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1051/2114] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:09 2022] DELETE /identity/v3/users/308361e94fbc47d099f2922994c5c0dc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:09.921596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:09.921883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:09.922161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:09.941825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:09.943040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:10.002575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1064/2115] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:10.013013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.014426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:10.073801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4db59b-0c6a-4b7e-96f1-96659686534b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.077517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb0d25-edac-4057-b4be-459cbb8cb05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.078812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb0d25-edac-4057-b4be-459cbb8cb05f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.078812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb0d25-edac-4057-b4be-459cbb8cb05f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.078812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb0d25-edac-4057-b4be-459cbb8cb05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.093035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb0d25-edac-4057-b4be-459cbb8cb05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.094644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb0d25-edac-4057-b4be-459cbb8cb05f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.107380 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1052/2116] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:10.396727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.398441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:10.412081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f3f24-be63-49ce-80ad-ac891d4f57f3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.417346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.418087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.418483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.418789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] PATH_INFO: `/v3/projects/e37fda2e939149409275a3ad0053a262` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.419971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37fda2e939149409275a3ad0053a262)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.421902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.474196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37fda2e939149409275a3ad0053a262', 'request_id': 'req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:10.485525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37fda2e939149409275a3ad0053a262', 'request_id': 'req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:10.494700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37fda2e939149409275a3ad0053a262', 'request_id': 'req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:10.507417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] The token cache is being invalidate because project e37fda2e939149409275a3ad0053a262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:10.507753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:10.508726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b0e9f283-eb69-48ce-bf2b-bea25f8957ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:10.510666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1065/2117] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:10 2022] DELETE /identity/v3/projects/e37fda2e939149409275a3ad0053a262 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:10.515423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.515917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.515998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.539272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:10.540519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:10.620891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1053/2118] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:10.630859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.631148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.631419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.651821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:10.653232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:10.661167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1066/2119] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:10.665347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.675660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1bb19-8290-4ecb-850f-fe98b790cc8f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.680134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.680725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.680999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.681267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.681905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:10.683718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.683858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.697005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1054/2120] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:10.701656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.712054 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1b7f76-cccd-46f4-a158-1bd13d0c487a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.717479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.718213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.718564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.718992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.720209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.721635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.755731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1067/2121] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:10 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:10.757347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.772651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8317b549-05e5-49af-b7d2-d9933f693839 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.842797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:10.857124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.858630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.858922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.859199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.859890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.861911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.897486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1055/2122] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:10 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:10.899115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.911844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4062c6e-e611-4f19-b5a9-75d491c9f5af admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.917265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.918374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.918664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.919184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.919945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.921626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.937687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1068/2123] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:10.939827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.951936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6c470-0515-4821-82a1-8e9b40c98dc7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.957281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74678c5e-3d11-499f-addc-57d890718045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.957906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678c5e-3d11-499f-addc-57d890718045 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.958186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678c5e-3d11-499f-addc-57d890718045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.958617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74678c5e-3d11-499f-addc-57d890718045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:10.959282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678c5e-3d11-499f-addc-57d890718045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:10.960918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74678c5e-3d11-499f-addc-57d890718045 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:10.977746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1056/2124] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:10.981837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:10.993026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdf556-7036-48cc-962b-f1502eec8465 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:10.998448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:10.998940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:10.999616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:10.999864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] PATH_INFO: `/v3/projects/8ddee21bebfc48c7b4b68e7a6697121c/users/49c1690956464e0ea1e8be74575b26ea/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.017987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ddee21bebfc48c7b4b68e7a6697121c, user_id=49c1690956464e0ea1e8be74575b26ea, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.021433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.037395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1069/2125] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:10 2022] PUT /identity/v3/projects/8ddee21bebfc48c7b4b68e7a6697121c/users/49c1690956464e0ea1e8be74575b26ea/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:11.048830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.049448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.049836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.078997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] MFA Rules not processed for user `49c1690956464e0ea1e8be74575b26ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:11.080375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:11.147355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1057/2126] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:11.151829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.176496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:11.223595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28997f85-63bb-4741-866f-4af81df9082d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.228603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.229216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.229496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.229766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.230401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.232369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.260459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1070/2127] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:11 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:11.264255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.276418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc3b41c-aeb4-46ea-b03a-5828ec68d85f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.342181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.342908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.343261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.343648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.344369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.346491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.372990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1058/2128] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:11 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:11.374284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.385318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d6e90b-50a6-4f58-aef4-c382d2a8fb6b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.390144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.390768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.391049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.391321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.392717 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.394340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.406044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1071/2129] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:11.410828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.421339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f5299-d09b-4f6c-9375-3927374be520 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.426074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.426803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.427055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.427330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] PATH_INFO: `/v3/projects/8042f27b285b4e588e649ef4a919e2ec/users/a14bf436d2d84f10ba78602207aba5cb/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.443011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] RBAC: Authorizing `identity:create_grant(project_id=8042f27b285b4e588e649ef4a919e2ec, user_id=a14bf436d2d84f10ba78602207aba5cb, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.445880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.460285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1059/2130] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:11 2022] PUT /identity/v3/projects/8042f27b285b4e588e649ef4a919e2ec/users/a14bf436d2d84f10ba78602207aba5cb/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:11.464209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.475010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7bf82-34ba-4f7f-b331-56a7f0dcc05f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.533377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864277be-2009-4461-a7ef-b47cfac471b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.534157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864277be-2009-4461-a7ef-b47cfac471b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.534447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864277be-2009-4461-a7ef-b47cfac471b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.534806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864277be-2009-4461-a7ef-b47cfac471b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.535770 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864277be-2009-4461-a7ef-b47cfac471b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.538663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864277be-2009-4461-a7ef-b47cfac471b9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.550018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1072/2131] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:11.554398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.565777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94ce3-1ce4-4af4-af13-ae897605dcec admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.570797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.571460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.571777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.572060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] PATH_INFO: `/v3/projects/8042f27b285b4e588e649ef4a919e2ec/users/a14bf436d2d84f10ba78602207aba5cb/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.574027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8042f27b285b4e588e649ef4a919e2ec, user_id=a14bf436d2d84f10ba78602207aba5cb, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.576812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.595590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1060/2132] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:11 2022] PUT /identity/v3/projects/8042f27b285b4e588e649ef4a919e2ec/users/a14bf436d2d84f10ba78602207aba5cb/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:11.602326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.602616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.603120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.632787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] MFA Rules not processed for user `a14bf436d2d84f10ba78602207aba5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:11.634012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:11.701906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1073/2133] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:11.712793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:11.714508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:11.729913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:11.772699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde1ab3-9f90-493e-a085-13c51b161b55 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:11.776513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5f9a5d-b4ec-4826-93aa-9d0d709fe7b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:11.777124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f9a5d-b4ec-4826-93aa-9d0d709fe7b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:11.777408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f9a5d-b4ec-4826-93aa-9d0d709fe7b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:11.777687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f9a5d-b4ec-4826-93aa-9d0d709fe7b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:11.833468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f9a5d-b4ec-4826-93aa-9d0d709fe7b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:11.834903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5f9a5d-b4ec-4826-93aa-9d0d709fe7b2 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:11.843839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1061/2134] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:57:15.162765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:15.164199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:15.178408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:15.229797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa00c4-3ea7-4ec1-b19e-9ca0c9e2d614 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:15.234287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:15.234917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:15.235219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:15.235496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] PATH_INFO: `/v3/users/ba225ed230cf48a79d5e5acc424b157e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:15.236649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba225ed230cf48a79d5e5acc424b157e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:15.238557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:15.289750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba225ed230cf48a79d5e5acc424b157e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.296246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba225ed230cf48a79d5e5acc424b157e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.308517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba225ed230cf48a79d5e5acc424b157e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.309152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba225ed230cf48a79d5e5acc424b157e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.309767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba225ed230cf48a79d5e5acc424b157e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.321608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1074/2135] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:15 2022] DELETE /identity/v3/users/ba225ed230cf48a79d5e5acc424b157e => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:15.326108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:15.326547 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:15.326639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:15.346525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:15.347783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:15.406644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1062/2136] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:15.416464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:15.418036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:15.484867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9f102-4fe8-4d4f-a60c-59a6632f023e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:15.488714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:15.489400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:15.489823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:15.490158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:15.499797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:15.501736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:15.512162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1075/2137] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:15.852182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:15.853664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:15.865475 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a330660e-9a7e-4fe0-9f8a-662713a90036 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:15.870120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:15.870755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:15.871034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:15.871306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] PATH_INFO: `/v3/projects/de82c2f32eb4464a9a7cb139ab8a1b76` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:15.872465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=de82c2f32eb4464a9a7cb139ab8a1b76)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:15.874377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:15.926889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de82c2f32eb4464a9a7cb139ab8a1b76', 'request_id': 'req-834d2550-2230-404e-89f4-a36c4cbdc56b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.934316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de82c2f32eb4464a9a7cb139ab8a1b76', 'request_id': 'req-834d2550-2230-404e-89f4-a36c4cbdc56b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.939236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:15.940659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de82c2f32eb4464a9a7cb139ab8a1b76', 'request_id': 'req-834d2550-2230-404e-89f4-a36c4cbdc56b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.958223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] The token cache is being invalidate because project de82c2f32eb4464a9a7cb139ab8a1b76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:15.958508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.959183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:15.960702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1063/2138] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:15 2022] DELETE /identity/v3/projects/de82c2f32eb4464a9a7cb139ab8a1b76 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:15.964880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:15.975322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf83886-9d22-4855-84ef-443a0760437c service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.008205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:16.035936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.036573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.036859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.037137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] PATH_INFO: `/v3/users/d8a85d8812ab49c0bcff379b170bd2a6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.038271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8a85d8812ab49c0bcff379b170bd2a6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.040256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.084839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8a85d8812ab49c0bcff379b170bd2a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.091930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8a85d8812ab49c0bcff379b170bd2a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.104710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8a85d8812ab49c0bcff379b170bd2a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.105337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8a85d8812ab49c0bcff379b170bd2a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.105962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-084d059e-07e7-4a9e-8891-1a03e62f32e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8a85d8812ab49c0bcff379b170bd2a6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.117982 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1076/2139] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:15 2022] DELETE /identity/v3/users/d8a85d8812ab49c0bcff379b170bd2a6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:16.402992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.404738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:16.418409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d2550-2230-404e-89f4-a36c4cbdc56b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.482701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.483416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.483804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.484156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] PATH_INFO: `/v3/projects/248e24fd1f2549639337c182f244809f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.485951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=248e24fd1f2549639337c182f244809f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.488109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.543723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '248e24fd1f2549639337c182f244809f', 'request_id': 'req-d92ef243-b48b-40d3-a0d4-e32421a84887'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.552163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '248e24fd1f2549639337c182f244809f', 'request_id': 'req-d92ef243-b48b-40d3-a0d4-e32421a84887'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.559015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '248e24fd1f2549639337c182f244809f', 'request_id': 'req-d92ef243-b48b-40d3-a0d4-e32421a84887'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.571271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] The token cache is being invalidate because project 248e24fd1f2549639337c182f244809f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:16.571805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.572562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d92ef243-b48b-40d3-a0d4-e32421a84887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:16.574363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1064/2140] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:16 2022] DELETE /identity/v3/projects/248e24fd1f2549639337c182f244809f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:16.579712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.580132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.580132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.600262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:16.601726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:16.669374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1077/2141] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:16.677099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.677389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.677666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.697323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:16.698524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:16.706877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1065/2142] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:16.711302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.721741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e3fec-d7bd-43d4-bac6-78e1931d9e00 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.726623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.727224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.727506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.727811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.728456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:16.728764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.730410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.741853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1078/2143] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:16.746087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.755388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1a08f-4739-438d-8da2-9124cd393979 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.760020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.760608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.760885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.761156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.761781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.763689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.791733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1066/2144] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:16 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:16.795828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.805735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e738c07-e3ee-4c7a-8d2e-30649c8f4631 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.846503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:16.865915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.866532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.866811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.867082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.867753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.869669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.892828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1079/2145] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:16 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:16.897098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.908075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecd55c-77ed-4c04-acd2-217fc0ba6c52 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.912846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.913514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.913878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.914147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.914867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.916764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.928836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1067/2146] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:16.932903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.945352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef2ab7-1285-495a-95eb-a28ea02af859 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.950228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3ded89-95cf-4c15-b141-6013c8183ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.950945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ded89-95cf-4c15-b141-6013c8183ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.951237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ded89-95cf-4c15-b141-6013c8183ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.951511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3ded89-95cf-4c15-b141-6013c8183ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:16.952209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ded89-95cf-4c15-b141-6013c8183ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:16.954796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3ded89-95cf-4c15-b141-6013c8183ec6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:16.967195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1080/2147] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:16 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:16.971693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:16.982683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6056f4-2a7e-4021-916a-8baec9eb14ff admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:16.987555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:16.988370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:16.988716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:16.989059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] PATH_INFO: `/v3/projects/ac4bca95f73544e68559017cfb12af0f/users/914ca14ee6114a07b102a4e01d43cdd9/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.008363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4bca95f73544e68559017cfb12af0f, user_id=914ca14ee6114a07b102a4e01d43cdd9, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.011899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.027251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1068/2148] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:16 2022] PUT /identity/v3/projects/ac4bca95f73544e68559017cfb12af0f/users/914ca14ee6114a07b102a4e01d43cdd9/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:17.039317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.039703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.039979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.069655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] MFA Rules not processed for user `914ca14ee6114a07b102a4e01d43cdd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:17.070949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:17.144951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1081/2149] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:17.149824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.174148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:17.229531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04261549-6217-477e-a035-bc0a73c4fe2d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.234396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.234994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.235842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.236116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.236754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.238666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.265089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1069/2150] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:17 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:17.269639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.284780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819edcb0-ddfc-45dd-8a24-d5ca64fc7b59 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.346139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.346788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.347071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.347350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.348037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.350006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.373329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1082/2151] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:17 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:17.377386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.388255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381415d6-63a5-403f-afdf-76bffdcb363e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.393331 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.393929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.394204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.394472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.395105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.396722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.407692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1070/2152] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:17.412101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.422473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607213bb-ccd3-464f-ae29-3d32497c8840 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.428882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.430046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.430497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.430786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] PATH_INFO: `/v3/projects/c619162e1d31431a9ccf15e29c16efc4/users/ab2a295c0db743d6a7b2827a0a7c06e3/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.446619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c619162e1d31431a9ccf15e29c16efc4, user_id=ab2a295c0db743d6a7b2827a0a7c06e3, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.449341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.462870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1083/2153] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:17 2022] PUT /identity/v3/projects/c619162e1d31431a9ccf15e29c16efc4/users/ab2a295c0db743d6a7b2827a0a7c06e3/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:17.467016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.478462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957daa8-46b7-4f54-b2ac-24eff9b347da admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.538932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d292c-d2e0-42c5-a7c2-1b9158cf4274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.539563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d292c-d2e0-42c5-a7c2-1b9158cf4274 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.539845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d292c-d2e0-42c5-a7c2-1b9158cf4274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.540113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d292c-d2e0-42c5-a7c2-1b9158cf4274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.540813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d292c-d2e0-42c5-a7c2-1b9158cf4274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.542424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d292c-d2e0-42c5-a7c2-1b9158cf4274 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.552731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1071/2154] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:17.557382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.569218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ae6fc8-4974-4c2f-b0ad-d7e2e2fd7baa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.575940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.576599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.576881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.577152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] PATH_INFO: `/v3/projects/c619162e1d31431a9ccf15e29c16efc4/users/ab2a295c0db743d6a7b2827a0a7c06e3/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.579672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c619162e1d31431a9ccf15e29c16efc4, user_id=ab2a295c0db743d6a7b2827a0a7c06e3, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.582379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.593873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1084/2155] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:17 2022] PUT /identity/v3/projects/c619162e1d31431a9ccf15e29c16efc4/users/ab2a295c0db743d6a7b2827a0a7c06e3/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:17.606172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.606454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.606730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.641258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] MFA Rules not processed for user `ab2a295c0db743d6a7b2827a0a7c06e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:17.643060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:17.725029 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1072/2156] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:17 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:17.738977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.740836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:17.746260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:17.800158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca097955-64b5-4d94-8412-e27b773aedfa admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.803917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.804624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.804898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.805191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.814826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.816294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.825911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1085/2157] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:17.902214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:17.912774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cff62-f237-41e2-be42-42fd7fa42157 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:17.916428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5371556-4445-42ab-902f-46e736c68b03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:17.917031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5371556-4445-42ab-902f-46e736c68b03 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:17.917305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5371556-4445-42ab-902f-46e736c68b03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:17.917574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5371556-4445-42ab-902f-46e736c68b03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:17.927069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:17.979604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5371556-4445-42ab-902f-46e736c68b03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:17.981002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5371556-4445-42ab-902f-46e736c68b03 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:17.990431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1073/2158] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:57:22.098741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:22.100513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.174834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925446d-6931-4d60-b9cc-191cb831ac1a service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:22.180047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:22.180701 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.181018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.181264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] PATH_INFO: `/v3/users/49c1690956464e0ea1e8be74575b26ea` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.183110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c1690956464e0ea1e8be74575b26ea)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:22.185150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:22.248262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c1690956464e0ea1e8be74575b26ea'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.256686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c1690956464e0ea1e8be74575b26ea'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.270182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c1690956464e0ea1e8be74575b26ea'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.270882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c1690956464e0ea1e8be74575b26ea'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.271518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c1690956464e0ea1e8be74575b26ea'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.285341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1086/2159] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:22 2022] DELETE /identity/v3/users/49c1690956464e0ea1e8be74575b26ea => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:22.290073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.290532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.290787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.310604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:22.311891 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.372878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1074/2160] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:22.383578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:22.384998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.417315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:22.425081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:22.426552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.438536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457a82d-ab2a-4ce6-af2d-63dcf1273a4f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:22.443309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:22.444043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.444322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.444592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] PATH_INFO: `/v3/users/15957a08541d4ee993dd3b4632fde187` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.445770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15957a08541d4ee993dd3b4632fde187)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:22.447740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:22.450273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b518e71-8751-403e-916b-8da53482cd7d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:22.454568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0a1ce-1382-48d3-8dae-b06d5a7556c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:22.455241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0a1ce-1382-48d3-8dae-b06d5a7556c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.455491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0a1ce-1382-48d3-8dae-b06d5a7556c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.455780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0a1ce-1382-48d3-8dae-b06d5a7556c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.465219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0a1ce-1382-48d3-8dae-b06d5a7556c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:22.466647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0a1ce-1382-48d3-8dae-b06d5a7556c0 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:22.477972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1087/2161] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:22.502543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15957a08541d4ee993dd3b4632fde187'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.510099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15957a08541d4ee993dd3b4632fde187'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.510722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15957a08541d4ee993dd3b4632fde187'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.524592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15957a08541d4ee993dd3b4632fde187'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.525451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15957a08541d4ee993dd3b4632fde187'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.539220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1075/2162] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:22 2022] DELETE /identity/v3/users/15957a08541d4ee993dd3b4632fde187 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:22.544643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.544999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.545533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.567565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:22.568874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.640338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1088/2163] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:22.649952 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:22.651888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.681356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:22.714367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b3abb-2afe-4e76-b8b5-d38e73fa0e84 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:22.718789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:22.719447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.719765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.720033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.744622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:22.746301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:22.757886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1076/2164] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:22.758434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:22.760377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:22.779963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3228f7-f52c-4005-8fe9-ebc3c9e36f33 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:22.788652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:22.789758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.789969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.790307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] PATH_INFO: `/v3/projects/8ddee21bebfc48c7b4b68e7a6697121c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.791515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ddee21bebfc48c7b4b68e7a6697121c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:22.793779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:22.845134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ddee21bebfc48c7b4b68e7a6697121c', 'request_id': 'req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.854092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ddee21bebfc48c7b4b68e7a6697121c', 'request_id': 'req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.863349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ddee21bebfc48c7b4b68e7a6697121c', 'request_id': 'req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.875286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] The token cache is being invalidate because project 8ddee21bebfc48c7b4b68e7a6697121c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:22.876796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.877469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:22.879182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1089/2165] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:22 2022] DELETE /identity/v3/projects/8ddee21bebfc48c7b4b68e7a6697121c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:22.883062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:22.893683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396ba35-face-4401-a8f7-a6823091e0a7 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:22.959896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:22.960467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:22.960740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:22.961009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] PATH_INFO: `/v3/users/a14bf436d2d84f10ba78602207aba5cb` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:22.962209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a14bf436d2d84f10ba78602207aba5cb)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:22.964310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.020564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a14bf436d2d84f10ba78602207aba5cb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.033791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a14bf436d2d84f10ba78602207aba5cb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.034796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a14bf436d2d84f10ba78602207aba5cb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.055952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.057700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.069453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a14bf436d2d84f10ba78602207aba5cb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.071567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a14bf436d2d84f10ba78602207aba5cb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.076705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974c7c85-8bdd-4bfc-96c1-9d6ed9f3d351 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.081840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.082480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.082757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.083026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] PATH_INFO: `/v3/projects/94f50f8251964f149a94e25397797e6a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.084213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f50f8251964f149a94e25397797e6a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.086299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.101687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1077/2166] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:22 2022] DELETE /identity/v3/users/a14bf436d2d84f10ba78602207aba5cb => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:23.139320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:23.144769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f50f8251964f149a94e25397797e6a', 'request_id': 'req-252b5cea-0c9f-4297-b73c-cf437dcc061e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.165070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f50f8251964f149a94e25397797e6a', 'request_id': 'req-252b5cea-0c9f-4297-b73c-cf437dcc061e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.177684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f50f8251964f149a94e25397797e6a', 'request_id': 'req-252b5cea-0c9f-4297-b73c-cf437dcc061e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.192517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] The token cache is being invalidate because project 94f50f8251964f149a94e25397797e6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:23.192807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.193524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.195367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1090/2167] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:23 2022] DELETE /identity/v3/projects/94f50f8251964f149a94e25397797e6a => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:23.199585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.201005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.256896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:23.278934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715d5107-1410-4694-9f5b-06801ac6d6d5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.342318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.343093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.343397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.343718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] PATH_INFO: `/v3/users/9767feb1a012487dbae3b1bc9505b011` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.345139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9767feb1a012487dbae3b1bc9505b011)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.347218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.397925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9767feb1a012487dbae3b1bc9505b011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.405246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9767feb1a012487dbae3b1bc9505b011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.406164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9767feb1a012487dbae3b1bc9505b011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.415809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.417363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.436467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9767feb1a012487dbae3b1bc9505b011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.437479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b0641759-9cc1-4d49-bb35-9c739f807076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9767feb1a012487dbae3b1bc9505b011'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.438619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252b5cea-0c9f-4297-b73c-cf437dcc061e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.445781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.446585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.447147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.447424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] PATH_INFO: `/v3/projects/8042f27b285b4e588e649ef4a919e2ec` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.448896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8042f27b285b4e588e649ef4a919e2ec)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.451132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.452003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1078/2168] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:23 2022] DELETE /identity/v3/users/9767feb1a012487dbae3b1bc9505b011 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:23.514491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8042f27b285b4e588e649ef4a919e2ec', 'request_id': 'req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.524247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8042f27b285b4e588e649ef4a919e2ec', 'request_id': 'req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.533291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8042f27b285b4e588e649ef4a919e2ec', 'request_id': 'req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.548163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] The token cache is being invalidate because project 8042f27b285b4e588e649ef4a919e2ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:23.548460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.549130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8a24cb5d-4e8c-4eb5-8e94-c117bb124214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.550773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1091/2169] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:23 2022] DELETE /identity/v3/projects/8042f27b285b4e588e649ef4a919e2ec => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:23.556015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.556306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.556578 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.577723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:23.579064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.648485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1079/2170] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:23.656329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c03533-3771-4756-931c-be575efc0f99 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.656619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c03533-3771-4756-931c-be575efc0f99 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.657442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c03533-3771-4756-931c-be575efc0f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.676950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-10c03533-3771-4756-931c-be575efc0f99 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:23.678166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-10c03533-3771-4756-931c-be575efc0f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.691430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1092/2171] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:23.692779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.704207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26131a45-2dc1-41c8-a3cd-f84700c8c79b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.709445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.710089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.710345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.710620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.711579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:23.711813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.713580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.723567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:23.731736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1080/2172] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:23.732661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c03533-3771-4756-931c-be575efc0f99 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.743080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c03533-3771-4756-931c-be575efc0f99 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.748402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.749006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.749322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.749607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.750352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.752546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.761901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.763897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.780232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6093f97d-8ef9-49ba-bf1b-cbadd46e032e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.782723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1093/2173] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:23 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:23.786615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.843823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.845270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.845585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.845869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] PATH_INFO: `/v3/projects/ed157ffb648c4a109a241b0f37bcc1fd` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.847018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed157ffb648c4a109a241b0f37bcc1fd)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.848969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.850905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557fa49c-f0ea-49fd-b697-64c58789e133 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.856517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.857390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.857390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.857679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.858298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.860870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.882653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1094/2174] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:23 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:23.886623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.896335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbfa578-9e2c-46bb-a0f3-3829c53f952e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:23.897193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed157ffb648c4a109a241b0f37bcc1fd', 'request_id': 'req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.905375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed157ffb648c4a109a241b0f37bcc1fd', 'request_id': 'req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.909740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=11,keystone:SELECT=83,keystone:INSERT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:23.912516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed157ffb648c4a109a241b0f37bcc1fd', 'request_id': 'req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.939324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] The token cache is being invalidate because project ed157ffb648c4a109a241b0f37bcc1fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:23.939657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.940436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:23.942959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1081/2175] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:23 2022] DELETE /identity/v3/projects/ed157ffb648c4a109a241b0f37bcc1fd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:23.946708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.948096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.962444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:23.963128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:23.963411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:23.963719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:23.964369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:23.965974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:23.976291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1095/2176] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:57:23.980324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:23.982040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:23.995823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac47f1-0dc4-4f62-957d-33c14528c5ae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.000696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.001328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.001620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.001986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.002629 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.004316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.014658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1096/2177] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:57:24.020700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91efc40d-cc5c-4476-bcf5-61d5eeaf5647 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.020953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:24.025653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.026292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.026571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.026844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] PATH_INFO: `/v3/users/47bae3ce49b943d984a2a65b29c7ead6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.028026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47bae3ce49b943d984a2a65b29c7ead6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.029951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.031077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f308c-20f1-4597-b789-95a41643e8d5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.036687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e155fd87-ede6-4ed7-ab51-08ef7ae8dc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.037335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155fd87-ede6-4ed7-ab51-08ef7ae8dc2d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.037702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155fd87-ede6-4ed7-ab51-08ef7ae8dc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.037987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e155fd87-ede6-4ed7-ab51-08ef7ae8dc2d admin admin] PATH_INFO: `/v3/projects/fef86e1086f544be8a12e0057ae0b9b8/users/d1fb03d4dca24d49ac4443eebe544f38/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.055962 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155fd87-ede6-4ed7-ab51-08ef7ae8dc2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef86e1086f544be8a12e0057ae0b9b8, user_id=d1fb03d4dca24d49ac4443eebe544f38, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.058909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e155fd87-ede6-4ed7-ab51-08ef7ae8dc2d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.070366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:24.074748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1097/2178] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:24 2022] PUT /identity/v3/projects/fef86e1086f544be8a12e0057ae0b9b8/users/d1fb03d4dca24d49ac4443eebe544f38/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:24.085857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47bae3ce49b943d984a2a65b29c7ead6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.088084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a243684-1520-40fb-b46a-85477059a77f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.088372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a243684-1520-40fb-b46a-85477059a77f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.088645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a243684-1520-40fb-b46a-85477059a77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.094324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47bae3ce49b943d984a2a65b29c7ead6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.095444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47bae3ce49b943d984a2a65b29c7ead6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.108769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47bae3ce49b943d984a2a65b29c7ead6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.109471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47bae3ce49b943d984a2a65b29c7ead6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.118440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7a243684-1520-40fb-b46a-85477059a77f None None] MFA Rules not processed for user `d1fb03d4dca24d49ac4443eebe544f38`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:24.119814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7a243684-1520-40fb-b46a-85477059a77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:24.123177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1082/2179] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:23 2022] DELETE /identity/v3/users/47bae3ce49b943d984a2a65b29c7ead6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:24.207631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1098/2180] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:24.223190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:24.224747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:24.290114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dd5a5-3e06-4260-8c5d-9d2cbe3bd411 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.294595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ef545-8bb2-4d80-bdff-48cf089ad313 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.295210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef545-8bb2-4d80-bdff-48cf089ad313 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.295896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef545-8bb2-4d80-bdff-48cf089ad313 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.295896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ef545-8bb2-4d80-bdff-48cf089ad313 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.383490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ef545-8bb2-4d80-bdff-48cf089ad313 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.390086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ef545-8bb2-4d80-bdff-48cf089ad313 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.407431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1083/2181] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:24.487573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a243684-1520-40fb-b46a-85477059a77f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:24.489238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7a243684-1520-40fb-b46a-85477059a77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:24.566866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a243684-1520-40fb-b46a-85477059a77f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.577981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:24.628592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.629233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.629513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.629785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] PATH_INFO: `/v3/projects/2380162a06bc4cbc8ed96a38ac4d94be` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.631029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2380162a06bc4cbc8ed96a38ac4d94be)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.633433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.687877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2380162a06bc4cbc8ed96a38ac4d94be', 'request_id': 'req-38acb626-7999-4e7b-8af5-7a678a3dbb17'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.696693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2380162a06bc4cbc8ed96a38ac4d94be', 'request_id': 'req-38acb626-7999-4e7b-8af5-7a678a3dbb17'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.703133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2380162a06bc4cbc8ed96a38ac4d94be', 'request_id': 'req-38acb626-7999-4e7b-8af5-7a678a3dbb17'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.714118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] The token cache is being invalidate because project 2380162a06bc4cbc8ed96a38ac4d94be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:24.714408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.715089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-38acb626-7999-4e7b-8af5-7a678a3dbb17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:24.716624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1099/2182] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:24 2022] DELETE /identity/v3/projects/2380162a06bc4cbc8ed96a38ac4d94be => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:24.722065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07381c28-146d-4d4d-b094-48819534d496 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.722347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07381c28-146d-4d4d-b094-48819534d496 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.722618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07381c28-146d-4d4d-b094-48819534d496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.744590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-07381c28-146d-4d4d-b094-48819534d496 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:24.745724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-07381c28-146d-4d4d-b094-48819534d496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:24.809898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1084/2183] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:24.817940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.818227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.818507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.840740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:24.842129 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:24.850897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1100/2184] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:24.855183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07381c28-146d-4d4d-b094-48819534d496 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:24.866151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:24.873796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07381c28-146d-4d4d-b094-48819534d496 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.882046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.883041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.883323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.883619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.884271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:24.884656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.886851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.913027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1085/2185] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:24.914040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:24.946950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9038a83-cda4-42c2-8553-4c3cd79be00e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:24.952033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:24.952651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:24.952934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:24.953208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:24.954602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:24.956568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:24.994226 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1101/2186] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:24 2022] POST /identity/v3/projects => generated 371 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:24.998280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:25.008836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcebb6-bab9-445d-8dfd-40d7f1bfc95a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:25.073502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:25.074112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:25.074395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:25.074672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:25.075317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:25.077268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:25.098407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1086/2187] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:24 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:25.102906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:25.113869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747e1c97-be65-48f7-997c-9677aff09d49 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:25.118831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:25.119463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:25.119777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:25.120059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:25.120721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:25.122360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:25.132590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1102/2188] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:25.136434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:25.146361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4df15f-69b7-4d90-bfec-48518bed84a6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:25.151023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b9891-a309-4b19-b12c-b252b196b4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:25.151903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9891-a309-4b19-b12c-b252b196b4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:25.151903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9891-a309-4b19-b12c-b252b196b4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:25.152182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b9891-a309-4b19-b12c-b252b196b4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:25.152824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b9891-a309-4b19-b12c-b252b196b4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:25.154498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580b9891-a309-4b19-b12c-b252b196b4e3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:25.166021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1087/2189] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:25.170456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:25.181182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1846a9-3701-4a68-8ebb-3e60e4940f67 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:25.185858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:25.186505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:25.186788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:25.187069 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] PATH_INFO: `/v3/projects/4ebf52e3718140d1b2e4883f8d5464f4/users/72a393cb6b2343d697ec2e87d750c70a/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:25.201443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ebf52e3718140d1b2e4883f8d5464f4, user_id=72a393cb6b2343d697ec2e87d750c70a, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:25.204215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:25.216707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1103/2190] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:25 2022] PUT /identity/v3/projects/4ebf52e3718140d1b2e4883f8d5464f4/users/72a393cb6b2343d697ec2e87d750c70a/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:25.222297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46933c40-959a-4300-ae9a-b22a82348894 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:25.222590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46933c40-959a-4300-ae9a-b22a82348894 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:25.222893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46933c40-959a-4300-ae9a-b22a82348894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:25.242089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-46933c40-959a-4300-ae9a-b22a82348894 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:25.243297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-46933c40-959a-4300-ae9a-b22a82348894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:25.300956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1088/2191] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:25.311173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:25.312389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:25.361083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:25.374376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5b6d7-5669-4517-8227-254ffa7291f3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:25.378712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:25.379245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:25.379524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:25.379909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:25.390742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:25.393898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:25.405771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1104/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:26.106449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1592254-9ca4-4996-a141-49bcb2e616e9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:26.106987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1592254-9ca4-4996-a141-49bcb2e616e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:26.106987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1592254-9ca4-4996-a141-49bcb2e616e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:26.138957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f1592254-9ca4-4996-a141-49bcb2e616e9 None None] MFA Rules not processed for user `72a393cb6b2343d697ec2e87d750c70a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:26.140447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f1592254-9ca4-4996-a141-49bcb2e616e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:26.214212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:26.223911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1089/2193] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:26.232881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:26.234501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:26.292696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b2dc9-811e-4ece-90cb-57b47d0cbfc1 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:26.296300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:26.296907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:26.297190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:26.298007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:26.306149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:26.307610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:26.316966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1105/2194] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:36.312506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=33 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:39.391558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea78b-f36a-4896-a878-7228f62e7266 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:39.392059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea78b-f36a-4896-a878-7228f62e7266 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:39.392159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ea78b-f36a-4896-a878-7228f62e7266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:39.411958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-381ea78b-f36a-4896-a878-7228f62e7266 None None] MFA Rules not processed for user `72a393cb6b2343d697ec2e87d750c70a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:39.413099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-381ea78b-f36a-4896-a878-7228f62e7266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:39.420350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1090/2195] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:43.386499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:43.388097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:43.423790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ad90e-c3a5-4649-9027-ac19f53cec7b service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:43.432686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:43.434713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:43.434713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:43.434713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] PATH_INFO: `/v3/users/d36682108c7040bfa841ea0dd1f634be` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:43.436389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36682108c7040bfa841ea0dd1f634be)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:43.441219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:43.501677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36682108c7040bfa841ea0dd1f634be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:43.510270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36682108c7040bfa841ea0dd1f634be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:43.524098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36682108c7040bfa841ea0dd1f634be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:43.524797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36682108c7040bfa841ea0dd1f634be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:43.527926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36682108c7040bfa841ea0dd1f634be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:43.557474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1106/2196] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:43 2022] DELETE /identity/v3/users/d36682108c7040bfa841ea0dd1f634be => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:43.561053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:43.563073 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:43.567730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:43.617339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:43.619456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:43.704473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1091/2197] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:43.715158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:43.716890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:43.780172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ef343-b93f-41ad-baf2-3891fd66e93f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:43.784030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:43.784659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:43.784947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:43.785220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:43.794480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:43.795959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:43.806141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1107/2198] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:57:44.055831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:44.057356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:44.070986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4feac-cf42-43d7-bb35-66fe87e8f01c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:44.082580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:44.083394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:44.083929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:44.084388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] PATH_INFO: `/v3/users/914ca14ee6114a07b102a4e01d43cdd9` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:44.085753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914ca14ee6114a07b102a4e01d43cdd9)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:44.087945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:44.146720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914ca14ee6114a07b102a4e01d43cdd9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.157235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914ca14ee6114a07b102a4e01d43cdd9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.158288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914ca14ee6114a07b102a4e01d43cdd9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.163478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:44.165552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:44.184847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914ca14ee6114a07b102a4e01d43cdd9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.185918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-66c9f3ee-5846-4251-8ac9-11c8f036ff39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914ca14ee6114a07b102a4e01d43cdd9'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.186368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e65e728-b7f7-476d-8d2c-39481b6cf902 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:44.192024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:44.192669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:44.192959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:44.193218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] PATH_INFO: `/v3/projects/4a0f587828dc4ae6980ed70a20291381` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:44.194347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a0f587828dc4ae6980ed70a20291381)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:44.196755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:44.208031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1092/2199] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:44 2022] DELETE /identity/v3/users/914ca14ee6114a07b102a4e01d43cdd9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:44.225054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:44.228283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:44.228640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:44.267102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a0f587828dc4ae6980ed70a20291381', 'request_id': 'req-db4ba55d-3464-4946-974b-fe1b044c7885'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.273279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:44.274722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:44.275363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a0f587828dc4ae6980ed70a20291381', 'request_id': 'req-db4ba55d-3464-4946-974b-fe1b044c7885'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.282937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a0f587828dc4ae6980ed70a20291381', 'request_id': 'req-db4ba55d-3464-4946-974b-fe1b044c7885'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.295033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:44.297874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] The token cache is being invalidate because project 4a0f587828dc4ae6980ed70a20291381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:44.298877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.299688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.302158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1108/2200] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:44 2022] DELETE /identity/v3/projects/4a0f587828dc4ae6980ed70a20291381 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:44.375340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1093/2201] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:44.410234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:44.411758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:44.518704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ba55d-3464-4946-974b-fe1b044c7885 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:44.522749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:44.523369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:44.523798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:44.524081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:44.534350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:44.535818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:44.545241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1109/2202] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:44.861547 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:44.863036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:44.876392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf2cf-1afd-48cf-8d39-0a8ff3320c39 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:44.881209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:44.882018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:44.882350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:44.882631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] PATH_INFO: `/v3/projects/ac4bca95f73544e68559017cfb12af0f` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:44.883875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4bca95f73544e68559017cfb12af0f)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:44.885855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:44.888312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:44.940538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4bca95f73544e68559017cfb12af0f', 'request_id': 'req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.949152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4bca95f73544e68559017cfb12af0f', 'request_id': 'req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.955052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4bca95f73544e68559017cfb12af0f', 'request_id': 'req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.967248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] The token cache is being invalidate because project ac4bca95f73544e68559017cfb12af0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:44.967569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.968315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:44.970732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1094/2203] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:44 2022] DELETE /identity/v3/projects/ac4bca95f73544e68559017cfb12af0f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:44.974707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:44.985851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12304983-52d0-4a05-b406-cc9306b10a9b service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.047784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.048387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.048669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.048969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] PATH_INFO: `/v3/users/ab2a295c0db743d6a7b2827a0a7c06e3` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.050268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab2a295c0db743d6a7b2827a0a7c06e3)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.052347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.103423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab2a295c0db743d6a7b2827a0a7c06e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.111336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab2a295c0db743d6a7b2827a0a7c06e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.123271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab2a295c0db743d6a7b2827a0a7c06e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.123951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab2a295c0db743d6a7b2827a0a7c06e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.124592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-bb2ed659-b261-4f72-b1d1-76ae986d3699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab2a295c0db743d6a7b2827a0a7c06e3'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.137478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1110/2204] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:44 2022] DELETE /identity/v3/users/ab2a295c0db743d6a7b2827a0a7c06e3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:45.408799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:45.410251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:45.424676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5cda7-dc65-4e6f-91ea-d90d6f0d98cc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.481406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.482034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.482310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.482579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] PATH_INFO: `/v3/projects/c619162e1d31431a9ccf15e29c16efc4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.483754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c619162e1d31431a9ccf15e29c16efc4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.486194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.533631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c619162e1d31431a9ccf15e29c16efc4', 'request_id': 'req-05152909-4138-4128-aecd-2817648a72eb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.542190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c619162e1d31431a9ccf15e29c16efc4', 'request_id': 'req-05152909-4138-4128-aecd-2817648a72eb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.549640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c619162e1d31431a9ccf15e29c16efc4', 'request_id': 'req-05152909-4138-4128-aecd-2817648a72eb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.560943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] The token cache is being invalidate because project c619162e1d31431a9ccf15e29c16efc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:45.561235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.561893 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-05152909-4138-4128-aecd-2817648a72eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:45.563397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1095/2205] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:45 2022] DELETE /identity/v3/projects/c619162e1d31431a9ccf15e29c16efc4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:45.568415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.568878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.569106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.588685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:45.589978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:45.604066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:45.668980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1111/2206] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:45.671101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.671389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.676553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.699528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:45.700804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:45.720973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1096/2207] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:45.721070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:45.726345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec843ae-85dc-41ca-a3a4-78b88e3dc492 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.732952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.733669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.734071 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.734984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.735800 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:45.736182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.737901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.750919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1112/2208] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:57:45.757210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:45.767272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a320e0-78a2-439d-91fd-03e780f038f5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.771932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.772534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.772816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.773085 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.773714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.775635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.787212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:45.804137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1097/2209] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:45 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:45.808371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:45.818586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593f41a-2599-4041-bdf7-65374c210a48 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.881821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.882891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.883283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.883588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.884292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.886280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.911559 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1113/2210] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:45 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:45.916133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:45.926726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2988e8-b9c1-48d5-8f36-1ef7db08adb1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.931337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.931979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.932260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.932533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.933298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.934925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.946229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1098/2211] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:45.951407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:45.962300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc23d27-cd2d-4bc4-a9c5-fc918a2937f8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:45.966871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a5c92-de97-44fa-beb4-3785c741d5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:45.967471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5c92-de97-44fa-beb4-3785c741d5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:45.967781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5c92-de97-44fa-beb4-3785c741d5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:45.968053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a5c92-de97-44fa-beb4-3785c741d5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:45.969278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5c92-de97-44fa-beb4-3785c741d5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:45.971104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a5c92-de97-44fa-beb4-3785c741d5d2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:45.986841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1114/2212] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:57:45.991215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.001991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cab73d-9d9b-424c-a81e-a1b91c0dc62e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.006724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.007391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.007707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.007980 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] PATH_INFO: `/v3/projects/f21d390392ef4bf1b8c46b3dbc0fdc82/users/c7e2d5231d294ed1a827b0b067262898/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.023831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f21d390392ef4bf1b8c46b3dbc0fdc82, user_id=c7e2d5231d294ed1a827b0b067262898, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.026491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.040295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1099/2213] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:45 2022] PUT /identity/v3/projects/f21d390392ef4bf1b8c46b3dbc0fdc82/users/c7e2d5231d294ed1a827b0b067262898/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:46.050617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.050934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.051206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.080376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] MFA Rules not processed for user `c7e2d5231d294ed1a827b0b067262898`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:46.081571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:46.157957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1115/2214] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:46.164083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.227922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43741566-e1c7-4bc5-b4d9-6a5689807274 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.232700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.233302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.233579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.233848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.234515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.236466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.274378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1100/2215] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:46 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:46.278305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.289675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b338127-3dfc-4644-a3df-33bc4b9e7de4 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.302529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:46.357108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.357753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.358070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.358578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.360109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.362152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.385735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1116/2216] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:46 2022] POST /identity/v3/users => generated 290 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:46.389820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.401549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b678c-71d2-4faa-b8c0-b400272d8ac5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.407342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.407983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.408375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.408656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.409313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.411243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.424903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1101/2217] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:46.429855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.440478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1de600-4822-4cd0-b162-bcd29c06bb2e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.445085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.445772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.446090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.446413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] PATH_INFO: `/v3/projects/5d05a01fad724f57ba983b6156d0a6cb/users/2d770bcb96874d47a167675dedb7a116/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.463719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d05a01fad724f57ba983b6156d0a6cb, user_id=2d770bcb96874d47a167675dedb7a116, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.466437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.480668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1117/2218] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:46 2022] PUT /identity/v3/projects/5d05a01fad724f57ba983b6156d0a6cb/users/2d770bcb96874d47a167675dedb7a116/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:46.485163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.495020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4875eb51-62ba-495b-9d0e-2969d79878b0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.557801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9183fe-dd02-4f6a-b293-2dd14cf11e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.558408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9183fe-dd02-4f6a-b293-2dd14cf11e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.558684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9183fe-dd02-4f6a-b293-2dd14cf11e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.558953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9183fe-dd02-4f6a-b293-2dd14cf11e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.559615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9183fe-dd02-4f6a-b293-2dd14cf11e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.561192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9183fe-dd02-4f6a-b293-2dd14cf11e2d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.570589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1102/2219] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:46 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:46.574726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.585417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3febf60-5688-4e1e-afeb-d8af078b4b1c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.590259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.590935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.591259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.591616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] PATH_INFO: `/v3/projects/5d05a01fad724f57ba983b6156d0a6cb/users/2d770bcb96874d47a167675dedb7a116/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.594174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d05a01fad724f57ba983b6156d0a6cb, user_id=2d770bcb96874d47a167675dedb7a116, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.596967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.608143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1118/2220] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:46 2022] PUT /identity/v3/projects/5d05a01fad724f57ba983b6156d0a6cb/users/2d770bcb96874d47a167675dedb7a116/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:46.619006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.619302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.619601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.647715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] MFA Rules not processed for user `2d770bcb96874d47a167675dedb7a116`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:46.648922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:46.662394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:46.719842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1103/2221] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:46.730777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:46.732532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:46.830237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfb17e-3894-4756-9652-ec952523ea37 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:46.834575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d6bd9-5ee2-4c60-8b4e-652abd5a6103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:46.835317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6bd9-5ee2-4c60-8b4e-652abd5a6103 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:46.835759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6bd9-5ee2-4c60-8b4e-652abd5a6103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:46.836324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d6bd9-5ee2-4c60-8b4e-652abd5a6103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:46.848312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d6bd9-5ee2-4c60-8b4e-652abd5a6103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:46.849978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d6bd9-5ee2-4c60-8b4e-652abd5a6103 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:46.861452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1119/2222] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:48.395099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:48.459943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b203e1e-0bbc-48f2-bc3b-ebec1f01f520 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:48.464562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:48.465195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:48.465476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:48.465751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] PATH_INFO: `/v3/users/c7e2d5231d294ed1a827b0b067262898` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:48.466898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e2d5231d294ed1a827b0b067262898)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:48.468899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:48.538101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e2d5231d294ed1a827b0b067262898'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:48.550218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e2d5231d294ed1a827b0b067262898'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:48.552200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e2d5231d294ed1a827b0b067262898'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:48.564900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e2d5231d294ed1a827b0b067262898'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:48.565572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e2d5231d294ed1a827b0b067262898'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:48.577518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1104/2223] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:48 2022] DELETE /identity/v3/users/c7e2d5231d294ed1a827b0b067262898 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:48.582702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:48.583171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:48.583281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:48.606389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:48.608871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:48.672355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1120/2224] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:48.681904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:48.683287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:48.717860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:48.745904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d460dc5-4e48-40a1-befd-bf3c1104aa44 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:48.750286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:48.750905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:48.751180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:48.751444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:48.761239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:48.762636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:48.772592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1105/2225] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:49.036501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:49.037922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:49.050060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a0b1e8-69f1-4da8-9bfc-5154afe02fb6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:49.055449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:49.056122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.056404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.056666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] PATH_INFO: `/v3/projects/f21d390392ef4bf1b8c46b3dbc0fdc82` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.057792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f21d390392ef4bf1b8c46b3dbc0fdc82)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:49.059726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:49.067850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:49.109716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f21d390392ef4bf1b8c46b3dbc0fdc82', 'request_id': 'req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.117232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f21d390392ef4bf1b8c46b3dbc0fdc82', 'request_id': 'req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.127171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f21d390392ef4bf1b8c46b3dbc0fdc82', 'request_id': 'req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.138980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] The token cache is being invalidate because project f21d390392ef4bf1b8c46b3dbc0fdc82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:49.139360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.140061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.141443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1121/2226] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:49 2022] DELETE /identity/v3/projects/f21d390392ef4bf1b8c46b3dbc0fdc82 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:49.145356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:49.155229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349ff1-f987-44af-b5ea-908b150654ad service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:49.221902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:49.222539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.222821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.223097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] PATH_INFO: `/v3/users/2d770bcb96874d47a167675dedb7a116` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.224430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d770bcb96874d47a167675dedb7a116)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:49.226759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:49.280117 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d770bcb96874d47a167675dedb7a116'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.289153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d770bcb96874d47a167675dedb7a116'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.289828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d770bcb96874d47a167675dedb7a116'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.303104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d770bcb96874d47a167675dedb7a116'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.303800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5e0f2a3d-1bf6-442f-b032-15a4c07abbce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d770bcb96874d47a167675dedb7a116'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.316179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1106/2227] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:49 2022] DELETE /identity/v3/users/2d770bcb96874d47a167675dedb7a116 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:49.590700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:49.592670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:49.607766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c3f58-1fae-4cd3-af52-9d3bd58ee9ed admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:49.667921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:49.668560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.668841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.669115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] PATH_INFO: `/v3/projects/5d05a01fad724f57ba983b6156d0a6cb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.670257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d05a01fad724f57ba983b6156d0a6cb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:49.672457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:49.720810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d05a01fad724f57ba983b6156d0a6cb', 'request_id': 'req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.728944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d05a01fad724f57ba983b6156d0a6cb', 'request_id': 'req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.735789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d05a01fad724f57ba983b6156d0a6cb', 'request_id': 'req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.746647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] The token cache is being invalidate because project 5d05a01fad724f57ba983b6156d0a6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:49.746937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.748233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f54b0bfc-b2fc-4f41-8dae-96a409e08fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:49.751634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1122/2228] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:49 2022] DELETE /identity/v3/projects/5d05a01fad724f57ba983b6156d0a6cb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:49.754257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.754770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.754919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.776505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:49.777716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:49.825898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:49.839650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1107/2229] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:49.850344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.850635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.851217 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.871995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:49.873216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:49.881835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1123/2230] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:49.886233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:49.896443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71191abc-b850-4ba9-84b7-178a1dc2ea05 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:49.901124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:49.901677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.901959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.902235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.902878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:49.903185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:49.904812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:49.916177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1108/2231] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:49.920239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:49.929263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3b71c-bd17-4045-9556-b74870aa1e3d None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:49.934060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:49.934660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:49.934958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:49.935219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:49.935904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:49.938349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:49.954521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:49.967591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1124/2232] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:49 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:49.971288 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:49.981756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac6b61f-87c7-4cef-8058-81a5808446c6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.044084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.044751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.045025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.045291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.045921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.048422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.071086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1109/2233] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:49 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:50.074994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.085987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d20b01-998c-4302-8a5e-83f4702e06b1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.090502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.091098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.091375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.091682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.092322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.093925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.104037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1125/2234] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:50.107865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.117627 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcecabd1-15aa-4b5c-83c0-a8558525d781 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.122187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffa8b7-f9ae-4a2a-9f20-7bbdbbd6b9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.122776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffa8b7-f9ae-4a2a-9f20-7bbdbbd6b9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.123056 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffa8b7-f9ae-4a2a-9f20-7bbdbbd6b9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.123329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffa8b7-f9ae-4a2a-9f20-7bbdbbd6b9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.124006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffa8b7-f9ae-4a2a-9f20-7bbdbbd6b9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.126130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffa8b7-f9ae-4a2a-9f20-7bbdbbd6b9c3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.138424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1110/2235] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:50.143086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.153087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f141a0-bc19-4572-b3b1-737b9113d326 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.157708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.158714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.159000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.159268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] PATH_INFO: `/v3/projects/e1167d82c5f3427f9ad2c663f0f25c0b/users/b164593c810a4835bc9c0be629ccab0b/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.173858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1167d82c5f3427f9ad2c663f0f25c0b, user_id=b164593c810a4835bc9c0be629ccab0b, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.176543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.190267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1126/2236] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:50 2022] PUT /identity/v3/projects/e1167d82c5f3427f9ad2c663f0f25c0b/users/b164593c810a4835bc9c0be629ccab0b/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:50.200424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.200715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.200991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.229290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] MFA Rules not processed for user `b164593c810a4835bc9c0be629ccab0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:50.230484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:50.302530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1111/2237] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:50.307037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.370062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdf79f-6a57-449b-a5d2-9fdc8ce3a97f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.375617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.376348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.376661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.376939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.377572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.379472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.409845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1127/2238] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:50 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:50.410992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.423661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea78996b-a390-42bf-a99a-f5d3f9195cf8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.475434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:50.492209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.492825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.493103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.493378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.494343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.496291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.518794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1112/2239] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:50 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:50.522967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.533250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aed349-c0bc-4351-b251-5e93b9d675b9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.537888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.538485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.538767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.539045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.539721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.541330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.552734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1128/2240] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:50.556928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.567495 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaa776-0baa-439f-ba0c-ea29eaa6430f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.572998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.573780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.574099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.574444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] PATH_INFO: `/v3/projects/e2955866590841549a5ed72d69edc465/users/dcab76a64d2242359081624518e6a21e/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.589574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2955866590841549a5ed72d69edc465, user_id=dcab76a64d2242359081624518e6a21e, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.593091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.621813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1113/2241] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:50 2022] PUT /identity/v3/projects/e2955866590841549a5ed72d69edc465/users/dcab76a64d2242359081624518e6a21e/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:50.623285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.633856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe65d41-210b-4b07-86cd-c85f508fd44c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.696434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475c1d1-ed36-4be8-80d5-6cf32bf83d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.697095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c1d1-ed36-4be8-80d5-6cf32bf83d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.697415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c1d1-ed36-4be8-80d5-6cf32bf83d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.697726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475c1d1-ed36-4be8-80d5-6cf32bf83d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.698399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475c1d1-ed36-4be8-80d5-6cf32bf83d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.700089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475c1d1-ed36-4be8-80d5-6cf32bf83d94 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.710350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1129/2242] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:50.715647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.728361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0c89e-1347-41ab-8e9c-516377255bd1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.738220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.743472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.743796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.744116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] PATH_INFO: `/v3/projects/e2955866590841549a5ed72d69edc465/users/dcab76a64d2242359081624518e6a21e/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.746227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2955866590841549a5ed72d69edc465, user_id=dcab76a64d2242359081624518e6a21e, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.749363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.760175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1114/2243] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:50 2022] PUT /identity/v3/projects/e2955866590841549a5ed72d69edc465/users/dcab76a64d2242359081624518e6a21e/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:50.770449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.770787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.771106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.800894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] MFA Rules not processed for user `dcab76a64d2242359081624518e6a21e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:50.802092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:50.821229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:50.879110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1130/2244] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:50.893127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:50.895041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:50.955737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8314b9f5-8988-4f84-8bf0-7a8fc1897fb0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:50.959361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ba9258-e026-4265-8394-956879905739 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:50.959991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9258-e026-4265-8394-956879905739 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:50.960268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9258-e026-4265-8394-956879905739 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:50.960539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ba9258-e026-4265-8394-956879905739 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:50.970133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba9258-e026-4265-8394-956879905739 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:50.971569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ba9258-e026-4265-8394-956879905739 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:50.981157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1115/2245] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:51.058860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:51.124549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9476b-ddce-429e-944a-f345fd5a0bbb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:51.129403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:51.130047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:51.130327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:51.130602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] PATH_INFO: `/v3/users/b164593c810a4835bc9c0be629ccab0b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:51.131783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b164593c810a4835bc9c0be629ccab0b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:51.133719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:51.181695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b164593c810a4835bc9c0be629ccab0b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.189004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b164593c810a4835bc9c0be629ccab0b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.200930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b164593c810a4835bc9c0be629ccab0b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.201564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b164593c810a4835bc9c0be629ccab0b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.202166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b164593c810a4835bc9c0be629ccab0b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.214633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1131/2246] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:51 2022] DELETE /identity/v3/users/b164593c810a4835bc9c0be629ccab0b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:51.219777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:51.220089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:51.220361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:51.238838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:51.240053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:51.297899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1116/2247] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:51.307457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:51.308952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:51.348525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:51.367177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea1c56e-c589-4eac-959f-cad6278898a6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:51.370780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:51.371384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:51.371708 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:51.372007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:51.381647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:51.383041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:51.397364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1132/2248] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:51.669564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:51.673148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:51.711585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e3304-91e5-4dc0-83fc-aefa8fcb2b6a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:51.716330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:51.717192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:51.717476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:51.718109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] PATH_INFO: `/v3/projects/e1167d82c5f3427f9ad2c663f0f25c0b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:51.724552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1167d82c5f3427f9ad2c663f0f25c0b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:51.730597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:51.779005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:51.788403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1167d82c5f3427f9ad2c663f0f25c0b', 'request_id': 'req-84b837a5-295f-4601-a55e-27606bc819c2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.803063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1167d82c5f3427f9ad2c663f0f25c0b', 'request_id': 'req-84b837a5-295f-4601-a55e-27606bc819c2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.809892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1167d82c5f3427f9ad2c663f0f25c0b', 'request_id': 'req-84b837a5-295f-4601-a55e-27606bc819c2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.822137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] The token cache is being invalidate because project e1167d82c5f3427f9ad2c663f0f25c0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:51.822433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.823100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.824644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1117/2249] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:51 2022] DELETE /identity/v3/projects/e1167d82c5f3427f9ad2c663f0f25c0b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:51.829437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:51.839695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0543bd-9c54-47be-95be-b5b83bf83327 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:51.901191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:51.901794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:51.902110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:51.902408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] PATH_INFO: `/v3/users/dcab76a64d2242359081624518e6a21e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:51.903595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcab76a64d2242359081624518e6a21e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:51.905568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:51.954213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcab76a64d2242359081624518e6a21e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.961910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcab76a64d2242359081624518e6a21e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.975581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcab76a64d2242359081624518e6a21e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.976825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcab76a64d2242359081624518e6a21e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.977583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-34da9a38-6552-4d3a-a71f-c449d89306f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcab76a64d2242359081624518e6a21e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:51.991396 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1133/2250] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:57:51 2022] DELETE /identity/v3/users/dcab76a64d2242359081624518e6a21e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:52.301788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.303265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:52.316531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b837a5-295f-4601-a55e-27606bc819c2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.374366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.375022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.375324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.375754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] PATH_INFO: `/v3/projects/e2955866590841549a5ed72d69edc465` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.376895 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2955866590841549a5ed72d69edc465)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.378788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.423830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2955866590841549a5ed72d69edc465', 'request_id': 'req-11fa796f-ad1d-4723-9046-66bb4974cb0c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:52.430823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2955866590841549a5ed72d69edc465', 'request_id': 'req-11fa796f-ad1d-4723-9046-66bb4974cb0c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:52.436709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2955866590841549a5ed72d69edc465', 'request_id': 'req-11fa796f-ad1d-4723-9046-66bb4974cb0c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:52.447349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] The token cache is being invalidate because project e2955866590841549a5ed72d69edc465 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:57:52.447673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:52.448295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-11fa796f-ad1d-4723-9046-66bb4974cb0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:57:52.449751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1118/2251] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:57:52 2022] DELETE /identity/v3/projects/e2955866590841549a5ed72d69edc465 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:52.454729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.455024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.455306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.475201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:52.476729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:52.526225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:52.537288 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1134/2252] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:52.544901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.545296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.545575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.565333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:52.566555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:52.574329 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1119/2253] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:52.578970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.588748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1248cb9-a23d-4ae3-8e48-398c29d77446 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.593916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.594554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.594834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.595108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.595798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:57:52.596110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.597705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.609400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1135/2254] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:57:52.613508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.622530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03f2ab4-e902-41e4-89e6-fe5c1ba83490 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.628032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.628633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.628959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.629235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.629864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.631823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.659617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1120/2255] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:52 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:52.663812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.674006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eff0ee7-ddef-42fe-9bbd-ba2e8aa1c1f6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.736956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.737583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.737880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.738128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.738759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.740729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.767350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1136/2256] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:52 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:52.772421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.787734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44332277-d0df-4ae4-bb09-22915fdc2105 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.793379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.794012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.794297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.794573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.795234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.797128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.811641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1121/2257] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:52.815499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.838614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1dcab-3568-41d4-af68-d51a3faadb49 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.843953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4736873-88aa-438c-aab5-6553aa6372b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.844582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736873-88aa-438c-aab5-6553aa6372b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.844867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736873-88aa-438c-aab5-6553aa6372b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.845160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4736873-88aa-438c-aab5-6553aa6372b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.845803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4736873-88aa-438c-aab5-6553aa6372b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.847424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4736873-88aa-438c-aab5-6553aa6372b9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.861266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1137/2258] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:57:52.864812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:52.877414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b5447-e3f7-4f6d-835a-40bbeacf674d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:52.883030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:52.883866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.884215 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.884630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] PATH_INFO: `/v3/projects/64a24f8c8cf24fb89bbbb4678ea70bdd/users/bd502edfb6c44037a75192b4098bf4d4/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.901856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a24f8c8cf24fb89bbbb4678ea70bdd, user_id=bd502edfb6c44037a75192b4098bf4d4, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:52.904711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:52.907021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:52.921466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1122/2259] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:52 2022] PUT /identity/v3/projects/64a24f8c8cf24fb89bbbb4678ea70bdd/users/bd502edfb6c44037a75192b4098bf4d4/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:57:52.931564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:52.931995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:52.932368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:52.961861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] MFA Rules not processed for user `bd502edfb6c44037a75192b4098bf4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:52.963367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:53.034176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1138/2260] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:53.038629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.114116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d3b19-a2a6-439e-84d5-248b4daa61e1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.118881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.119487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.119795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.120615 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.120731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.122685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.149768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1123/2261] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:53 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:53.153879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.166464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c111e-9b61-4739-9a13-438cbcc484a8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.229097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:53.240297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.241044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.241326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.241600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.242244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.244189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.268301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1139/2262] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:53 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:53.272347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.283248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b242d3-7eb2-4538-84cd-cbd9c955ae85 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.287898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.288492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.288768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.289151 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.289821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.291401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.302366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1124/2263] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:53.306060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.316384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadbb5e8-df3d-46b8-941a-2af24eaae38c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.321004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e171a-5b0a-4f0c-8b40-872886f56bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.321606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e171a-5b0a-4f0c-8b40-872886f56bcd admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.321888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e171a-5b0a-4f0c-8b40-872886f56bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.322177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e171a-5b0a-4f0c-8b40-872886f56bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.322796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e171a-5b0a-4f0c-8b40-872886f56bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.324416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e171a-5b0a-4f0c-8b40-872886f56bcd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.334880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1140/2264] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:53.339050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.349346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443fe966-66e2-4ec0-ae31-5db76423578e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.354670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.355321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.355636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.355927 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] PATH_INFO: `/v3/projects/3638e73ab0f842edb77b8f23599b9651/users/4356f2a56d4d4f6caa5c8b76562aa9ae/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.371995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3638e73ab0f842edb77b8f23599b9651, user_id=4356f2a56d4d4f6caa5c8b76562aa9ae, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.375277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.389761 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1125/2265] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:53 2022] PUT /identity/v3/projects/3638e73ab0f842edb77b8f23599b9651/users/4356f2a56d4d4f6caa5c8b76562aa9ae/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:53.401758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.402134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.402481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.431393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] MFA Rules not processed for user `4356f2a56d4d4f6caa5c8b76562aa9ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:53.432595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:53.496688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1141/2266] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:57:53.501907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.561549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64dc0c0-fd9e-4a20-87f0-6be683594660 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.566775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.567369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.567681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.567955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.568597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.570479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.596667 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1126/2267] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:57:53 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:57:53.601120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.612951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3045aa2-ea2d-46dd-9dc3-51941ff9a743 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.668831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.669437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.669718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.670100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.670806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.673395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.699463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1142/2268] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:57:53 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:57:53.701392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.712614 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b599ee69-85c5-4c78-9d96-593c0e7ff14b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.717608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.718751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.719034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.719313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.719994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.721654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.733130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1127/2269] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:53.737894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.748018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcca775-ff7c-45da-9a56-b38034748271 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.752751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.753416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.753697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.754015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] PATH_INFO: `/v3/projects/90fb7b13a0cc4fb4803bc46ccc1c55ec/users/a2652e65db2849f8ae87593ba6ad2b52/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.769309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=90fb7b13a0cc4fb4803bc46ccc1c55ec, user_id=a2652e65db2849f8ae87593ba6ad2b52, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.772172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.785755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1143/2270] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:53 2022] PUT /identity/v3/projects/90fb7b13a0cc4fb4803bc46ccc1c55ec/users/a2652e65db2849f8ae87593ba6ad2b52/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:53.789718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.800007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513e82d-4266-4f59-bd7d-ed464bf5b506 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.812546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:53.859612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22d8b0-f3ac-44f0-95ee-20229bbc3c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.860354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22d8b0-f3ac-44f0-95ee-20229bbc3c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.860692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22d8b0-f3ac-44f0-95ee-20229bbc3c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.861043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22d8b0-f3ac-44f0-95ee-20229bbc3c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.861752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22d8b0-f3ac-44f0-95ee-20229bbc3c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.863385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22d8b0-f3ac-44f0-95ee-20229bbc3c13 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.876878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1128/2271] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:57:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:57:53.878103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:53.888389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98502676-3148-445e-9a07-0d27e0f008bb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:53.893005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:53.893671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.893957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.894232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] PATH_INFO: `/v3/projects/90fb7b13a0cc4fb4803bc46ccc1c55ec/users/a2652e65db2849f8ae87593ba6ad2b52/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.896151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90fb7b13a0cc4fb4803bc46ccc1c55ec, user_id=a2652e65db2849f8ae87593ba6ad2b52, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:53.898765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:53.909239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1144/2272] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:57:53 2022] PUT /identity/v3/projects/90fb7b13a0cc4fb4803bc46ccc1c55ec/users/a2652e65db2849f8ae87593ba6ad2b52/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:57:53.919052 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:53.919346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:53.919661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:53.949809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] MFA Rules not processed for user `a2652e65db2849f8ae87593ba6ad2b52`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:57:53.950941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:54.030266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1129/2273] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:57:53 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:57:54.041950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:54.043690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:54.058988 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:57:54.103856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee5b8ca-1dda-4570-a89c-2ff3db6117c8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:54.107604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:54.108189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:54.108463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:54.109283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:54.118006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:54.119435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:54.129503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1145/2274] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:57:54.669765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:57:54.671187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:57:54.683106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f957131-39bf-4b94-806b-72c52ffbec21 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:57:54.686659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:57:54.687862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:57:54.688142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:57:54.688438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:57:54.744926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:57:54.746381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:57:54.756037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1130/2275] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:57:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:00.903091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:00.911786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1949454a-cc4e-42f2-9380-a27eed67a1f1 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:00.915373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a4a34-0f5d-4921-bd62-e9beffce3c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:00.916012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a4a34-0f5d-4921-bd62-e9beffce3c4d service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:00.916297 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a4a34-0f5d-4921-bd62-e9beffce3c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:00.916579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a4a34-0f5d-4921-bd62-e9beffce3c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:00.972769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a4a34-0f5d-4921-bd62-e9beffce3c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:00.974115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a4a34-0f5d-4921-bd62-e9beffce3c4d service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:00.984382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1146/2276] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:03.264838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:03.337054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:03.368592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0c6cb-a1a6-4875-8c2b-4db5a754909c service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:03.374899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:03.375575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:03.375850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:03.376303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] PATH_INFO: `/v3/users/bd502edfb6c44037a75192b4098bf4d4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:03.377448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd502edfb6c44037a75192b4098bf4d4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:03.379487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:03.435802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd502edfb6c44037a75192b4098bf4d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:03.448109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd502edfb6c44037a75192b4098bf4d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:03.448814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd502edfb6c44037a75192b4098bf4d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:03.463357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd502edfb6c44037a75192b4098bf4d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:03.464252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd502edfb6c44037a75192b4098bf4d4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:03.477409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1131/2277] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:03 2022] DELETE /identity/v3/users/bd502edfb6c44037a75192b4098bf4d4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:03.483948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:03.489721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:03.490213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:03.520674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:03.522035 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:03.633887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1147/2278] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:03.653033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:03.654638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:03.743459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7bec52-37bd-4df3-b772-110a299cb728 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:03.747377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:03.748149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:03.748921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:03.748921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:03.760345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:03.761789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:03.774393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1132/2279] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:04.157248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:04.158714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:04.172778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af69618-1bba-46c7-a401-04f07d4ef3bc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:04.179040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:04.179763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.180436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.180791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] PATH_INFO: `/v3/projects/64a24f8c8cf24fb89bbbb4678ea70bdd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.181961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a24f8c8cf24fb89bbbb4678ea70bdd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:04.184293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:04.235827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a24f8c8cf24fb89bbbb4678ea70bdd', 'request_id': 'req-8035da24-ae40-484c-93b2-8122c0559812'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.237388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:04.243966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a24f8c8cf24fb89bbbb4678ea70bdd', 'request_id': 'req-8035da24-ae40-484c-93b2-8122c0559812'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.251239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a24f8c8cf24fb89bbbb4678ea70bdd', 'request_id': 'req-8035da24-ae40-484c-93b2-8122c0559812'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.261689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] The token cache is being invalidate because project 64a24f8c8cf24fb89bbbb4678ea70bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:04.261978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.262627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.264146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1148/2280] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:04 2022] DELETE /identity/v3/projects/64a24f8c8cf24fb89bbbb4678ea70bdd => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:04.268846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:04.278878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3927c-fefb-4ce5-b3ac-5ae4e8615d39 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:04.334143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:04.334777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.335064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.335340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] PATH_INFO: `/v3/users/4356f2a56d4d4f6caa5c8b76562aa9ae` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.336516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4356f2a56d4d4f6caa5c8b76562aa9ae)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:04.338425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:04.339972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:04.348935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:04.350472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:04.364326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035da24-ae40-484c-93b2-8122c0559812 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:04.369144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:04.369778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.370059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.370339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] PATH_INFO: `/v3/users/3dfe18ccb6634691b719e692d131f4b7` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.371473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dfe18ccb6634691b719e692d131f4b7)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:04.373443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:04.388079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4356f2a56d4d4f6caa5c8b76562aa9ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.396278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4356f2a56d4d4f6caa5c8b76562aa9ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.396890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4356f2a56d4d4f6caa5c8b76562aa9ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.410479 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4356f2a56d4d4f6caa5c8b76562aa9ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.411189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ee2c049b-7b25-4d6e-9744-30f0560a40d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4356f2a56d4d4f6caa5c8b76562aa9ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.424473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dfe18ccb6634691b719e692d131f4b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.430376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1133/2281] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:04 2022] DELETE /identity/v3/users/4356f2a56d4d4f6caa5c8b76562aa9ae => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:04.436296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dfe18ccb6634691b719e692d131f4b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.452148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dfe18ccb6634691b719e692d131f4b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.452890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dfe18ccb6634691b719e692d131f4b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.453515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dfe18ccb6634691b719e692d131f4b7'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.470413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1149/2282] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:04 2022] DELETE /identity/v3/users/3dfe18ccb6634691b719e692d131f4b7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:04.474886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.475196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.475470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.497510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:04.498648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:04.563473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1134/2283] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:04.573223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:04.574633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:04.635181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e902a4-bec5-4c9e-bc81-ddf8f78d2203 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:04.638854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:04.639469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.640311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.640584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.649885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:04.651327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:04.661874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1150/2284] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:04.750195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:04.751762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:04.765901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717950bc-f238-4702-9a3d-bb51f9f77825 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:04.772425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:04.773094 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.773518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.773809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] PATH_INFO: `/v3/projects/3638e73ab0f842edb77b8f23599b9651` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.775967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=3638e73ab0f842edb77b8f23599b9651)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:04.779899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:04.840744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3638e73ab0f842edb77b8f23599b9651', 'request_id': 'req-565c6927-0404-4507-93ec-8227f490f8ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.849214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3638e73ab0f842edb77b8f23599b9651', 'request_id': 'req-565c6927-0404-4507-93ec-8227f490f8ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.856440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3638e73ab0f842edb77b8f23599b9651', 'request_id': 'req-565c6927-0404-4507-93ec-8227f490f8ae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.871505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] The token cache is being invalidate because project 3638e73ab0f842edb77b8f23599b9651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:04.871973 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.873601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:04.877207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1135/2285] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:04 2022] DELETE /identity/v3/projects/3638e73ab0f842edb77b8f23599b9651 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:04.881092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:04.894430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b7944e-5507-4823-a7ba-4509cead0e1c service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:04.944315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:04.956525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:04.957164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:04.957448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:04.957723 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] PATH_INFO: `/v3/users/a2652e65db2849f8ae87593ba6ad2b52` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:04.958859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2652e65db2849f8ae87593ba6ad2b52)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:04.960824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.007362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2652e65db2849f8ae87593ba6ad2b52'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.014181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2652e65db2849f8ae87593ba6ad2b52'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.027919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2652e65db2849f8ae87593ba6ad2b52'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.028614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2652e65db2849f8ae87593ba6ad2b52'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.029597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-799f81b3-edfa-4cd6-b44c-c9a9d0b4d2f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2652e65db2849f8ae87593ba6ad2b52'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.030942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.033023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.044095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:05.045682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1151/2286] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:04 2022] DELETE /identity/v3/users/a2652e65db2849f8ae87593ba6ad2b52 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:05.078822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565c6927-0404-4507-93ec-8227f490f8ae admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.224365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.225137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.225563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.226429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] PATH_INFO: `/v3/projects/8f934e9bb8f74b798e5e43f8a1975f5a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.230272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f934e9bb8f74b798e5e43f8a1975f5a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.235755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.356509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f934e9bb8f74b798e5e43f8a1975f5a', 'request_id': 'req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.366748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f934e9bb8f74b798e5e43f8a1975f5a', 'request_id': 'req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.392459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f934e9bb8f74b798e5e43f8a1975f5a', 'request_id': 'req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.413724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] The token cache is being invalidate because project 8f934e9bb8f74b798e5e43f8a1975f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:05.414112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.414750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cbe8f4ce-5262-4b23-ad48-92d1f783e38e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.416281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1136/2287] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:05 2022] DELETE /identity/v3/projects/8f934e9bb8f74b798e5e43f8a1975f5a => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:05.421831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.422341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.422774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.443036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:05.444302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.520948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.521464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.521794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.526868 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1152/2288] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:05.544885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:05.546340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.552765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.554775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.568872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1137/2289] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:05.571126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946dbf37-8c6a-4105-853f-3df88f491ecc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.574409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.576329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.576981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.577798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.578075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] PATH_INFO: `/v3/projects/90fb7b13a0cc4fb4803bc46ccc1c55ec` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.579447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] RBAC: Authorizing `identity:delete_project(project_id=90fb7b13a0cc4fb4803bc46ccc1c55ec)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.581430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.584663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e512ee6-85a2-47ca-bdf5-1cad5e1b7d68 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.591770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.592373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.592670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.593562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.594133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:05.594440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.596045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.609009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1138/2290] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:58:05.613597 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.623755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979c1ce-0b41-44c1-a5b8-4de0abc3034d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.629833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.630444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.630726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.631000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.631696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.633676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.660111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90fb7b13a0cc4fb4803bc46ccc1c55ec', 'request_id': 'req-edc2907c-25ed-4382-83d3-4e12cbbd32be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.665983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1139/2291] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:05.669994 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90fb7b13a0cc4fb4803bc46ccc1c55ec', 'request_id': 'req-edc2907c-25ed-4382-83d3-4e12cbbd32be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.670773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.679679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90fb7b13a0cc4fb4803bc46ccc1c55ec', 'request_id': 'req-edc2907c-25ed-4382-83d3-4e12cbbd32be'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.687682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:05.695318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] The token cache is being invalidate because project 90fb7b13a0cc4fb4803bc46ccc1c55ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:05.695853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.696752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-edc2907c-25ed-4382-83d3-4e12cbbd32be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:05.700335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1153/2292] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:05 2022] DELETE /identity/v3/projects/90fb7b13a0cc4fb4803bc46ccc1c55ec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:05.706875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32b510-8afa-46d1-b035-f9b81caaa673 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.707275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32b510-8afa-46d1-b035-f9b81caaa673 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.707673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c32b510-8afa-46d1-b035-f9b81caaa673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.728268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0c32b510-8afa-46d1-b035-f9b81caaa673 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:05.729588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0c32b510-8afa-46d1-b035-f9b81caaa673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.733270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:05.750947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaeef15-021d-4de3-a6e6-c61011516428 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.757384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.758003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.758289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.758714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.759494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.761612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.789084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1140/2293] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:05.790148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.791888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.793580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1154/2294] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:05.801651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.801850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.802285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.805030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48f0e68-2c02-4d14-a31e-113b8a97b412 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.809781 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.810863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.810863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.811041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.812768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.813889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.823192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:05.824438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:05.826083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1141/2295] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:05.830384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.833304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1155/2296] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:05.837754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.840569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a436a2d-3afd-4944-9813-fbba126930d2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.845394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.845985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.846265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.846536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.847151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29687b-c4e2-4ac6-b0bf-c830f1110bdb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.847269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.848847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.852451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.853194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.853478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.853813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.854501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:05.855381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.857192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.865863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1142/2297] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:05.870953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1156/2298] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:58:05.871279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.876856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.883898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c31d5-3ea6-48b0-9942-55ac570593c5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.887054 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40bf508-4ea4-45e6-954d-80063f5f2bf0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:05.891337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235245e-6c0b-472d-bb9d-08d3908db9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.892045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235245e-6c0b-472d-bb9d-08d3908db9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.892439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235245e-6c0b-472d-bb9d-08d3908db9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.892714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235245e-6c0b-472d-bb9d-08d3908db9e6 admin admin] PATH_INFO: `/v3/projects/6a76424101584e3c881c38eaa63586c2/users/aab868538c9b421a86cf6e03b31eff82/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.893017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:05.893658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.893943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.894219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.894882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.896838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.908290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235245e-6c0b-472d-bb9d-08d3908db9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a76424101584e3c881c38eaa63586c2, user_id=aab868538c9b421a86cf6e03b31eff82, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:05.910901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235245e-6c0b-472d-bb9d-08d3908db9e6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:05.929357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1157/2299] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:05.929766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1143/2300] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:05 2022] PUT /identity/v3/projects/6a76424101584e3c881c38eaa63586c2/users/aab868538c9b421a86cf6e03b31eff82/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:05.937080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:05.940296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:05.940584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:05.940862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:05.979166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] MFA Rules not processed for user `aab868538c9b421a86cf6e03b31eff82`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:05.980402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:06.013622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024816a-cbd5-4bc5-8303-f1e99863cde5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.023475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.024229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.024666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.024858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.025907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.029906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.060763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1144/2301] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:06.061485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1158/2302] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:05 2022] POST /identity/v3/users => generated 297 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:06.072048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.072237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.079377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:06.090765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159f93e-20ef-4f83-b88d-38882a175079 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.096797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.098383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.098383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.098577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.099490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.101623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.112962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1145/2303] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:06.116995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.127467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40a1878-d134-44a6-98d2-0a387d29fd44 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.134997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.134997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.135162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.135298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.136496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.138612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.153122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:06.155896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1146/2304] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:06.156908 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.171309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a4309-0744-4b15-b38f-e715b0508b58 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.177803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0dbf-c8ba-48af-8d1d-e6c66aab072d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.178558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0dbf-c8ba-48af-8d1d-e6c66aab072d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.179007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0dbf-c8ba-48af-8d1d-e6c66aab072d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.179353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0dbf-c8ba-48af-8d1d-e6c66aab072d admin admin] PATH_INFO: `/v3/projects/5a1ab11feecd42c695edc3c59b1a30bf/users/9eb4a1fe2c6e4083b6562136d0c01d79/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.182794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cee92e1-ba53-4802-b7a3-e6f3d9ed55ae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.188419 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.189219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.189704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.190530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.198633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce0dbf-c8ba-48af-8d1d-e6c66aab072d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1ab11feecd42c695edc3c59b1a30bf, user_id=9eb4a1fe2c6e4083b6562136d0c01d79, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.201415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.202374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bce0dbf-c8ba-48af-8d1d-e6c66aab072d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.202900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.214024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1159/2305] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:58:06.221034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1147/2306] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:06 2022] PUT /identity/v3/projects/5a1ab11feecd42c695edc3c59b1a30bf/users/9eb4a1fe2c6e4083b6562136d0c01d79/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:06.239642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.243313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.246904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.247164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.257294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042eaab9-0894-4dc4-a940-b93b666b76a4 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.308867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] MFA Rules not processed for user `9eb4a1fe2c6e4083b6562136d0c01d79`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:06.314166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:06.358135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.359198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.359791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.360653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] PATH_INFO: `/v3/users/aab868538c9b421a86cf6e03b31eff82` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.362301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab868538c9b421a86cf6e03b31eff82)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.364609 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.375242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:06.439266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab868538c9b421a86cf6e03b31eff82'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:06.443782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1148/2307] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:06.448577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.452186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab868538c9b421a86cf6e03b31eff82'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:06.474608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403027c-b35a-4a3c-a058-3a47c52dce01 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.477804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab868538c9b421a86cf6e03b31eff82'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:06.479198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab868538c9b421a86cf6e03b31eff82'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:06.480164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1139658c-fff5-4092-b890-ae90e52c48f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab868538c9b421a86cf6e03b31eff82'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:06.488181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.488821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.489106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.489383 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.490035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.493632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.499316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1160/2308] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:06 2022] DELETE /identity/v3/users/aab868538c9b421a86cf6e03b31eff82 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:06.508582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.508582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.508582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.527094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:06.528603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:06.546898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1149/2309] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:06 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:06.562454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.570555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:06.680694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1161/2310] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:06.703433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.706144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:06.723065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:06.749745 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea9f627-38f4-4ecd-8d57-a8b2f8e29862 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.766372 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.767008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.767282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.767598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.768488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.771864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.804087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1150/2311] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:06 2022] POST /identity/v3/users => generated 303 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:06.805542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.817081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f618a3-5c8f-4b43-8a36-ad4a983b203a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.829072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc83b81-d9a4-4df1-ac4e-a3dce873e62e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.845940 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.847305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.847867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.848402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.852124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.870491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.882759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.883368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.883710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.884037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.899734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.901209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1151/2312] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:06 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:06.901605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:06.907428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:06.921767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1162/2313] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:06.939323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff756c01-c2cd-4004-8feb-cc64aa9a4a26 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:06.944905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:06.945584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:06.945894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:06.946173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] PATH_INFO: `/v3/projects/d8859774a62e465ab34a57ba1614e9bb/users/d5d9899b119e418c9ac301d2aeacdbc6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:06.976764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8859774a62e465ab34a57ba1614e9bb, user_id=d5d9899b119e418c9ac301d2aeacdbc6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:06.979855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.019849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1152/2314] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:06 2022] PUT /identity/v3/projects/d8859774a62e465ab34a57ba1614e9bb/users/d5d9899b119e418c9ac301d2aeacdbc6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:07.020992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.033700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2859b447-f7ef-4bc6-ad8e-6ca09116b1cd service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.112747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf94f8-ef36-45db-b871-5391f707a4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.113386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf94f8-ef36-45db-b871-5391f707a4de admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.113667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf94f8-ef36-45db-b871-5391f707a4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.114083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf94f8-ef36-45db-b871-5391f707a4de admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.119996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf94f8-ef36-45db-b871-5391f707a4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.121839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf94f8-ef36-45db-b871-5391f707a4de admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.138861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1163/2315] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:07 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:07.143068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.157105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da0cee-1ea2-4e48-a278-9ba993606f13 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.170700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.171974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.172293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.172838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] PATH_INFO: `/v3/projects/d8859774a62e465ab34a57ba1614e9bb/users/d5d9899b119e418c9ac301d2aeacdbc6/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.175330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8859774a62e465ab34a57ba1614e9bb, user_id=d5d9899b119e418c9ac301d2aeacdbc6, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.181191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.195661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1153/2316] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:07 2022] PUT /identity/v3/projects/d8859774a62e465ab34a57ba1614e9bb/users/d5d9899b119e418c9ac301d2aeacdbc6/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:07.207125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.207490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.207777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.237510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] MFA Rules not processed for user `d5d9899b119e418c9ac301d2aeacdbc6`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:07.238751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.292689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.294589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.303295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:07.329249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1164/2317] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:07 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:07.340194 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.341705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.361658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:07.380734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b4593-e242-4905-ba6d-d9f741add1f5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.385999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.386620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.386899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.387169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] PATH_INFO: `/v3/projects/6a76424101584e3c881c38eaa63586c2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.388306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a76424101584e3c881c38eaa63586c2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.390393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.405282 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc48b5-f23d-488d-9942-d24d54dbb1a6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.409659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.410326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.410610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.411034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.423536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.425617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.439057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1165/2318] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:07 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:58:07.449679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a76424101584e3c881c38eaa63586c2', 'request_id': 'req-9a15316d-7c3a-4d32-85ad-6155fb093458'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:07.459873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a76424101584e3c881c38eaa63586c2', 'request_id': 'req-9a15316d-7c3a-4d32-85ad-6155fb093458'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:07.466468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a76424101584e3c881c38eaa63586c2', 'request_id': 'req-9a15316d-7c3a-4d32-85ad-6155fb093458'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:07.491019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] The token cache is being invalidate because project 6a76424101584e3c881c38eaa63586c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:07.491320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:07.491462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.492109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9a15316d-7c3a-4d32-85ad-6155fb093458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:07.493253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.493633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1154/2319] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:07 2022] DELETE /identity/v3/projects/6a76424101584e3c881c38eaa63586c2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:07.500685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10b84-b601-430f-80d7-2a772ef06cec None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.500975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10b84-b601-430f-80d7-2a772ef06cec None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.501251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f10b84-b601-430f-80d7-2a772ef06cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.517683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b5ff84-d106-4145-9e4c-9dbc5b18bf57 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.545278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-65f10b84-b601-430f-80d7-2a772ef06cec None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:07.547025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-65f10b84-b601-430f-80d7-2a772ef06cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.609887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.610547 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.610897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.611175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.613111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.646393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1155/2320] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:07.653851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.654458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.654945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.684320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:07.685592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.696210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1156/2321] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:07.701216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.710943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.713119 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.713747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ec804-634f-4a3a-8d47-855c492de97e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.720826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.721831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.722742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.722742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.723225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:07.723668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.724826 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1166/2322] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:58:07.739153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691c93c2-627a-47b6-a46b-c2fe9b166624 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.759882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1157/2323] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:58:07.765330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.778041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40498e53-df5a-47d3-8d37-0dd49fc4d777 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:07.783880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:07.784753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.784876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.785266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.786378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:07.788599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:07.844042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:07.844356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:07.844654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:07.860834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1167/2324] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:07 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:07.864596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:07.877247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] MFA Rules not processed for user `72a393cb6b2343d697ec2e87d750c70a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:07.878471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:07.916936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:07.978310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1158/2325] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:08.034771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9596f1-9c43-441e-8f3d-1ebc5d8cd95d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:08.047475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:08.048818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:08.049937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:08.050944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:08.052796 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:08.067568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:08.113973 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1168/2326] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:07 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:08.118013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:08.129972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:08.141840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613af58d-0c74-4d5b-a3a9-c39de3e62ee9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:08.156277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:08.157019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:08.157334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:08.157628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:08.158793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:08.161174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:08.195010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1159/2327] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:08.196525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:08.210664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ed69e-5a96-4eaf-ac5e-ec848007ad1f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:08.223640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5838580-06e4-4885-8786-04e80987ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:08.225141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838580-06e4-4885-8786-04e80987ba4f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:08.225538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838580-06e4-4885-8786-04e80987ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:08.226166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838580-06e4-4885-8786-04e80987ba4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:08.226942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5838580-06e4-4885-8786-04e80987ba4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:08.229589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5838580-06e4-4885-8786-04e80987ba4f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:08.244354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1169/2328] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:08.250393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:08.267320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b14fd3-de40-47e0-9a5b-37b89019bf5f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:08.274683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:08.276432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:08.276925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:08.277390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] PATH_INFO: `/v3/projects/0998ce41664645e6b2825dc435c8a4c2/users/055ae4012f414796a9a82e177ce43af7/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:08.296677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0998ce41664645e6b2825dc435c8a4c2, user_id=055ae4012f414796a9a82e177ce43af7, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:08.300140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:08.319020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1160/2329] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:08 2022] PUT /identity/v3/projects/0998ce41664645e6b2825dc435c8a4c2/users/055ae4012f414796a9a82e177ce43af7/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:08.330679 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2288-d44f-49bf-a257-d07cef846786 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:08.331128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2288-d44f-49bf-a257-d07cef846786 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:08.331483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2288-d44f-49bf-a257-d07cef846786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:08.363201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7ede2288-d44f-49bf-a257-d07cef846786 None None] MFA Rules not processed for user `055ae4012f414796a9a82e177ce43af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:08.364523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7ede2288-d44f-49bf-a257-d07cef846786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:08.440149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1170/2330] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:08.454935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:08.456932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:08.520607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae168087-9653-4cf9-a31e-b99ed0bbba37 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:08.524504 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:08.525125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:08.525402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:08.525672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:08.535922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:08.537386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:08.549465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1161/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:10.044666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48935b2a-156b-46a9-aa75-f4dd86dd2791 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:10.045257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48935b2a-156b-46a9-aa75-f4dd86dd2791 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:10.045365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48935b2a-156b-46a9-aa75-f4dd86dd2791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:10.069509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-48935b2a-156b-46a9-aa75-f4dd86dd2791 None None] MFA Rules not processed for user `055ae4012f414796a9a82e177ce43af7`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:10.070831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-48935b2a-156b-46a9-aa75-f4dd86dd2791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:10.084197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1171/2332] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:11.657989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:11.662370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:11.751671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcd951-6a5e-4f0b-b240-8fce9be66d16 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:11.757000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:11.757738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:11.758114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:11.758483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] PATH_INFO: `/v3/users/9eb4a1fe2c6e4083b6562136d0c01d79` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:11.760324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eb4a1fe2c6e4083b6562136d0c01d79)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:11.762747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:11.814899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eb4a1fe2c6e4083b6562136d0c01d79'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:11.822812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eb4a1fe2c6e4083b6562136d0c01d79'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:11.824994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eb4a1fe2c6e4083b6562136d0c01d79'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:11.852263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eb4a1fe2c6e4083b6562136d0c01d79'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:11.853229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eb4a1fe2c6e4083b6562136d0c01d79'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:11.856954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:11.875214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1162/2333] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:11 2022] DELETE /identity/v3/users/9eb4a1fe2c6e4083b6562136d0c01d79 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:11.879707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:11.880206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:11.880305 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:11.900748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:11.901983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:11.985902 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1172/2334] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:11.996293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:11.998530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:12.142043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be278f-fb58-4bf3-a788-ce1a73ffad42 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:12.146209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:12.146830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:12.147101 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:12.147364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:12.160308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:12.162736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:12.174498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1163/2335] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:58:12.637822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:12.640584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:12.675142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94349541-c092-43a4-a84b-a7ce9416d8d8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:12.684824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a4301-a719-4144-b778-711031357203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:12.685477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a4301-a719-4144-b778-711031357203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:12.685765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a4301-a719-4144-b778-711031357203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:12.686036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a4301-a719-4144-b778-711031357203 admin admin] PATH_INFO: `/v3/projects/5a1ab11feecd42c695edc3c59b1a30bf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:12.688999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a4301-a719-4144-b778-711031357203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1ab11feecd42c695edc3c59b1a30bf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:12.691839 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a4301-a719-4144-b778-711031357203 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:12.737790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:12.775408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1ab11feecd42c695edc3c59b1a30bf', 'request_id': 'req-e55a4301-a719-4144-b778-711031357203'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:12.786469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1ab11feecd42c695edc3c59b1a30bf', 'request_id': 'req-e55a4301-a719-4144-b778-711031357203'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:12.795110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1ab11feecd42c695edc3c59b1a30bf', 'request_id': 'req-e55a4301-a719-4144-b778-711031357203'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:12.808870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e55a4301-a719-4144-b778-711031357203 admin admin] The token cache is being invalidate because project 5a1ab11feecd42c695edc3c59b1a30bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:12.809243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:12.810302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:12.816816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:12.817577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1173/2336] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:12 2022] DELETE /identity/v3/projects/5a1ab11feecd42c695edc3c59b1a30bf => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:12.844646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9350647c-95f2-4ed4-9830-d1815cf2fea0 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:12.951729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:12.952683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:12.953079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:12.953462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] PATH_INFO: `/v3/users/d5d9899b119e418c9ac301d2aeacdbc6` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:12.955048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5d9899b119e418c9ac301d2aeacdbc6)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:12.957922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.019096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5d9899b119e418c9ac301d2aeacdbc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.032505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5d9899b119e418c9ac301d2aeacdbc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.033170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5d9899b119e418c9ac301d2aeacdbc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.049956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5d9899b119e418c9ac301d2aeacdbc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.050994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5d9899b119e418c9ac301d2aeacdbc6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.068172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1164/2337] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:12 2022] DELETE /identity/v3/users/d5d9899b119e418c9ac301d2aeacdbc6 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:13.070880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.072395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.095614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a4301-a719-4144-b778-711031357203 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.181772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.182505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.187585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.188133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] PATH_INFO: `/v3/users/de43a221bd8148b39eabc978b97f0e2a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.189577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de43a221bd8148b39eabc978b97f0e2a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.191857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.267874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de43a221bd8148b39eabc978b97f0e2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.279354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de43a221bd8148b39eabc978b97f0e2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.295601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de43a221bd8148b39eabc978b97f0e2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.296821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de43a221bd8148b39eabc978b97f0e2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.297677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de43a221bd8148b39eabc978b97f0e2a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.317927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1174/2338] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:13 2022] DELETE /identity/v3/users/de43a221bd8148b39eabc978b97f0e2a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:13.445795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.447505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.465013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a9ddf-1bc6-4899-a680-52b7cadde5c8 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.476251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:13.571471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.572403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.572672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.572937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] PATH_INFO: `/v3/projects/d8859774a62e465ab34a57ba1614e9bb` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.574060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8859774a62e465ab34a57ba1614e9bb)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.575992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.612368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.615209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.632837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b1cbd7-7670-44f9-8935-2dd33b0c99f1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.638556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.639843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.640234 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.640585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] PATH_INFO: `/v3/projects/5d24ce688df84f049775fa082eefcdce` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.642370 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d24ce688df84f049775fa082eefcdce)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.644602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.660490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8859774a62e465ab34a57ba1614e9bb', 'request_id': 'req-04545781-c355-41f1-9500-cf447af4b547'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.668506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8859774a62e465ab34a57ba1614e9bb', 'request_id': 'req-04545781-c355-41f1-9500-cf447af4b547'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.674354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8859774a62e465ab34a57ba1614e9bb', 'request_id': 'req-04545781-c355-41f1-9500-cf447af4b547'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.674496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:13.687833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] The token cache is being invalidate because project d8859774a62e465ab34a57ba1614e9bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:13.688160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.688747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-04545781-c355-41f1-9500-cf447af4b547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.690205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1165/2339] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:13 2022] DELETE /identity/v3/projects/d8859774a62e465ab34a57ba1614e9bb => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:13.696043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e37c7-1842-496b-8011-fd10baf16e9a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.696334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e37c7-1842-496b-8011-fd10baf16e9a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.696643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e37c7-1842-496b-8011-fd10baf16e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.704429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d24ce688df84f049775fa082eefcdce', 'request_id': 'req-c2e52987-0dab-41df-ac42-1bd889732c7e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.712721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d24ce688df84f049775fa082eefcdce', 'request_id': 'req-c2e52987-0dab-41df-ac42-1bd889732c7e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.718562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-de6e37c7-1842-496b-8011-fd10baf16e9a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:13.719838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-de6e37c7-1842-496b-8011-fd10baf16e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.719977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d24ce688df84f049775fa082eefcdce', 'request_id': 'req-c2e52987-0dab-41df-ac42-1bd889732c7e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.731392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] The token cache is being invalidate because project 5d24ce688df84f049775fa082eefcdce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:13.731713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.732351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c2e52987-0dab-41df-ac42-1bd889732c7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:13.733854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1175/2340] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:13 2022] DELETE /identity/v3/projects/5d24ce688df84f049775fa082eefcdce => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:13.739715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a6f72-09cb-47be-9b8a-a81bdf323873 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.740004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a6f72-09cb-47be-9b8a-a81bdf323873 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.740281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a6f72-09cb-47be-9b8a-a81bdf323873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.760846 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-449a6f72-09cb-47be-9b8a-a81bdf323873 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:13.762045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-449a6f72-09cb-47be-9b8a-a81bdf323873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.783932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1166/2341] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:13.793316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.793785 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.794192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.814654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:13.815920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.822845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1176/2342] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:13.824352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1167/2343] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:13.831426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.831694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.831825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.832029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.841506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416877b-291d-45f4-8ad2-a370fc5537a5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.846958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.847609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.847892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.848294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.848810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:13.849109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.850941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.853001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:13.854218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:13.862431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1177/2344] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:13.862896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1168/2345] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:58:13.866382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.866913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.877542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a02645-ca8d-4dff-a0d3-b20cbfef0a5b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.882441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.883034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.883308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.883601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.884228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.884330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f105f-ce55-43c2-9351-48a06ff0e327 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.886106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.889016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.890185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.890468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.890742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.891382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:13.891720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.893447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.905168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1178/2346] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:58:13.908774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:13.909951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.921509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1169/2347] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:13.922386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:13.924002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54e940-9b8e-4ef8-82fc-43d6c154a16b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.982125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.982732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.983030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.983313 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.983964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.985855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:13.986244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc725a15-3278-4fe3-8a47-d24a86f1b815 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:13.990931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:13.991532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:13.991843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:13.992118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:13.992748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:13.994638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.015465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1179/2348] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:14.020098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.021132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1170/2349] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:13 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:14.022012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.032300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a50733-9f41-4c70-9541-f5056a7129f6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.038372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:14.087487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e249cb68-bd06-4ab1-adcd-1f0fd3a08faf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.091377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.092014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.092192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.092305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.092576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.092829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.093114 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.093217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.093426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.094077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.094961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.096191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.110444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1171/2350] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:14.111964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.121580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1180/2351] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:14 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:14.125911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.129072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c95003-7ed1-4187-b351-d046c5a651c9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.133931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.134609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.134887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.135155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.135868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.136221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1cb65-746b-4cfb-ac1b-6b8bd15e45d0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.137498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.141564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.142166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.142544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.142830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.143505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.145712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.149251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1172/2352] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:14.153405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.164098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f67c5f-87a4-480a-a3dd-789e28aa16f6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.168619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1181/2353] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:14.169050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.169529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.169711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.169999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.170267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] PATH_INFO: `/v3/projects/1415ffebffae48cf9614823e445b90aa/users/a3c1dae7437a455ea225cf15b60e3f0b/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.185270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923c6168-8a65-49ee-afc9-05691b60e47b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.187066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1415ffebffae48cf9614823e445b90aa, user_id=a3c1dae7437a455ea225cf15b60e3f0b, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.189920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.190539 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60eb3bd-60de-41f3-9c93-55eaac0ad89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.191147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb3bd-60de-41f3-9c93-55eaac0ad89f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.191433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb3bd-60de-41f3-9c93-55eaac0ad89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.191751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60eb3bd-60de-41f3-9c93-55eaac0ad89f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.192510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eb3bd-60de-41f3-9c93-55eaac0ad89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.194154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60eb3bd-60de-41f3-9c93-55eaac0ad89f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.204293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1173/2354] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:14 2022] PUT /identity/v3/projects/1415ffebffae48cf9614823e445b90aa/users/a3c1dae7437a455ea225cf15b60e3f0b/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:14.205863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1182/2355] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:58:14.209990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.214764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.215057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.215334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.245230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] MFA Rules not processed for user `a3c1dae7437a455ea225cf15b60e3f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:14.246610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:14.261516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:14.274449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3a78f-551c-4bfd-9a14-729cdc560035 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.279587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53793-c486-4568-8a98-32dbce6c5d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.280829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53793-c486-4568-8a98-32dbce6c5d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.281109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53793-c486-4568-8a98-32dbce6c5d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.281385 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53793-c486-4568-8a98-32dbce6c5d75 admin admin] PATH_INFO: `/v3/projects/a446dbd47db84c4987a3ee5332356b84/users/2fcfbaf5bab14a4783ca912a222341c0/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.296225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c53793-c486-4568-8a98-32dbce6c5d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a446dbd47db84c4987a3ee5332356b84, user_id=2fcfbaf5bab14a4783ca912a222341c0, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.298831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c53793-c486-4568-8a98-32dbce6c5d75 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.312867 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1174/2356] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:14 2022] PUT /identity/v3/projects/a446dbd47db84c4987a3ee5332356b84/users/2fcfbaf5bab14a4783ca912a222341c0/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:14.315788 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1183/2357] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:14.317838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.318130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.318409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.326711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.328793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:14.337762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:14.338955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:14.391381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d5480-766c-4078-96ad-3913662c44ec None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.405295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1175/2358] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:14.415939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:14.417325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:14.429387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:14.457991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7e12dd-be05-45cb-b1a6-656457ba6417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.458610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e12dd-be05-45cb-b1a6-656457ba6417 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.458996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e12dd-be05-45cb-b1a6-656457ba6417 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.459280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7e12dd-be05-45cb-b1a6-656457ba6417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.494918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81c6797-5109-4497-bde2-6dcb1648c948 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:14.499706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:14.500351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:14.500698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:14.501042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:14.514362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.515852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.526031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1176/2359] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:58:14.531665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e12dd-be05-45cb-b1a6-656457ba6417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:14.533104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7e12dd-be05-45cb-b1a6-656457ba6417 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:14.549591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1184/2360] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 15:58:19.415492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:19.428500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02e6b9-35d3-4762-86f4-f208b64b1b56 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:19.434165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:19.434967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:19.435211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:19.435594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] PATH_INFO: `/v3/users/a3c1dae7437a455ea225cf15b60e3f0b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:19.437793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c1dae7437a455ea225cf15b60e3f0b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:19.439298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:19.496284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c1dae7437a455ea225cf15b60e3f0b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:19.506637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c1dae7437a455ea225cf15b60e3f0b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:19.507712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c1dae7437a455ea225cf15b60e3f0b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:19.529321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c1dae7437a455ea225cf15b60e3f0b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:19.530296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c1dae7437a455ea225cf15b60e3f0b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:19.531985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:19.556713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1177/2361] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:19 2022] DELETE /identity/v3/users/a3c1dae7437a455ea225cf15b60e3f0b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:19.558540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:19.559008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:19.559105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:19.605514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:19.607002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:19.691443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1185/2362] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:19.706574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:19.708375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:19.811807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20381d51-f992-43c9-a0b5-c5d97c83f816 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:19.822663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5966556b-7eba-4edd-a9df-e9d8850c88ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:19.823303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966556b-7eba-4edd-a9df-e9d8850c88ae service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:19.823673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966556b-7eba-4edd-a9df-e9d8850c88ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:19.823949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5966556b-7eba-4edd-a9df-e9d8850c88ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:19.851588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966556b-7eba-4edd-a9df-e9d8850c88ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:19.853278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5966556b-7eba-4edd-a9df-e9d8850c88ae service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:19.865786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1178/2363] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:20.569314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:20.571081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:20.591830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9412eb-42d3-4359-b349-d16d28309521 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:20.601914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:20.603522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:20.604152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:20.604630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] PATH_INFO: `/v3/projects/1415ffebffae48cf9614823e445b90aa` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:20.606104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1415ffebffae48cf9614823e445b90aa)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:20.608511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:20.684171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1415ffebffae48cf9614823e445b90aa', 'request_id': 'req-9907eab1-013a-4ce6-8905-60bfdb059a46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:20.697191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1415ffebffae48cf9614823e445b90aa', 'request_id': 'req-9907eab1-013a-4ce6-8905-60bfdb059a46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:20.706535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1415ffebffae48cf9614823e445b90aa', 'request_id': 'req-9907eab1-013a-4ce6-8905-60bfdb059a46'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:20.730381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] The token cache is being invalidate because project 1415ffebffae48cf9614823e445b90aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:20.730943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:20.731876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9907eab1-013a-4ce6-8905-60bfdb059a46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:20.733751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1186/2364] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:20 2022] DELETE /identity/v3/projects/1415ffebffae48cf9614823e445b90aa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:20.738797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:20.739831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:20.740116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:20.766337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:20.767990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:20.853060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1179/2365] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:20.861200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:20.861512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:20.861813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:20.890446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:20.897867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:20.907874 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:20.921203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1187/2366] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:20.922406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:20.933820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20cf3d-2ff6-43bb-991a-297637203a19 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:20.938926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:20.939752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:20.940138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:20.940718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:20.941499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:20.942066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:20.943821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f646d7-3175-4903-a3b1-93b6c9fec728 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:20.960350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1180/2367] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:58:20.964728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:20.977879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1af20f-721a-4795-a0b1-ef49c0508547 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:20.985577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:20.986384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:20.986736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:20.987082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:20.987851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:20.990050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.034729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.035265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.035724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.044455 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1188/2368] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:20 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:21.046074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.090554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] MFA Rules not processed for user `2fcfbaf5bab14a4783ca912a222341c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:21.092136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.131738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c668-88cd-481c-84d5-66a453e4fe82 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.137385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.138009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.138326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.140275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.141885 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.143873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.220344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1181/2369] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:21.251373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.255058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1189/2370] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:21 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:21.258382 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.265162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.282146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e08a01a-2632-46ca-9bae-0441df5472b8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.288801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:21.302172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.302915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.303198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.306915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.307594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.309503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.348883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1190/2371] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:21.350141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.352197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.382334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df756539-e306-4f0f-b37d-5a4539a329d3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.387263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d3be1-ccd0-460d-a4da-cbb59c2214e9 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.387631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.388632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.389087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.389970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] PATH_INFO: `/v3/users/d1fb03d4dca24d49ac4443eebe544f38` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.391771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1fb03d4dca24d49ac4443eebe544f38)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.394067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.395593 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.396419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.396710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.398861 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.410371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.412213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.440986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1182/2372] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:21.447784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.461999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1fb03d4dca24d49ac4443eebe544f38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:21.472058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1fb03d4dca24d49ac4443eebe544f38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:21.479824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6f7d09-1118-47fe-8aaa-c277dd5a05b4 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.486432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.487067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.487351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.488373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.489989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.491210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1fb03d4dca24d49ac4443eebe544f38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:21.491770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.492322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1fb03d4dca24d49ac4443eebe544f38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:21.494618 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-398e431e-a8e0-46a1-b636-79342a64c2b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1fb03d4dca24d49ac4443eebe544f38'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:21.508515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1183/2373] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:21 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:21.511708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.511913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1191/2374] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:21 2022] DELETE /identity/v3/users/d1fb03d4dca24d49ac4443eebe544f38 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:21.517518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.517810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.518092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.518308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.535460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f38c48-f8e5-4426-99bd-49d33e171500 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.556210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:21.557673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.595238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:21.659900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1192/2375] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:21.663991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ad64a-0c64-4288-8b09-b9e04b8df454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.664674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ad64a-0c64-4288-8b09-b9e04b8df454 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.665271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ad64a-0c64-4288-8b09-b9e04b8df454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.665271 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ad64a-0c64-4288-8b09-b9e04b8df454 admin admin] PATH_INFO: `/v3/projects/5c469a04e1114ad7a1761786485b294e/users/c4d688a00e5449499f08e7ece22ce159/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.666703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.668257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.701091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ad64a-0c64-4288-8b09-b9e04b8df454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c469a04e1114ad7a1761786485b294e, user_id=c4d688a00e5449499f08e7ece22ce159, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.704492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ad64a-0c64-4288-8b09-b9e04b8df454 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.723220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1184/2376] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:21 2022] PUT /identity/v3/projects/5c469a04e1114ad7a1761786485b294e/users/c4d688a00e5449499f08e7ece22ce159/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:21.752095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52ffeab-c1eb-496f-b75b-1c4d42d5ef98 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:21.798365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.798931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.798931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.836967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:21.837945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:21.838530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:21.838979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:21.844183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] MFA Rules not processed for user `c4d688a00e5449499f08e7ece22ce159`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:21.848431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:21.954978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:21.968562 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1185/2377] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:21.991909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:21.994475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:21.996236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:21.998640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:22.012151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1193/2378] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:22.163170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e41d2-ab2a-4b61-b944-fda77f89d00f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:22.169704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f6222-562d-489c-90f5-d4d3812ab03b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:22.170731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f6222-562d-489c-90f5-d4d3812ab03b service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:22.171202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f6222-562d-489c-90f5-d4d3812ab03b service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:22.171943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f6222-562d-489c-90f5-d4d3812ab03b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:22.203644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f6222-562d-489c-90f5-d4d3812ab03b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:22.226219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f6222-562d-489c-90f5-d4d3812ab03b service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:22.257914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1186/2379] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:22.902155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:22.906233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:22.931209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd26eae-5bba-4803-a2c0-3bfe95116b45 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:22.936602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:22.937245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:22.937526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:22.937799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] PATH_INFO: `/v3/projects/fef86e1086f544be8a12e0057ae0b9b8` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:22.938921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef86e1086f544be8a12e0057ae0b9b8)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:22.940877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.006466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:23.026528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef86e1086f544be8a12e0057ae0b9b8', 'request_id': 'req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:23.036744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef86e1086f544be8a12e0057ae0b9b8', 'request_id': 'req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:23.048264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef86e1086f544be8a12e0057ae0b9b8', 'request_id': 'req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:23.061598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] The token cache is being invalidate because project fef86e1086f544be8a12e0057ae0b9b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:23.061863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:23.062528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9285104c-6b6e-4cd6-8de2-ea85d72c68d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:23.067413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1194/2380] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:22 2022] DELETE /identity/v3/projects/fef86e1086f544be8a12e0057ae0b9b8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:23.069121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.069633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.069717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.111696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:23.113654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:23.237241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1187/2381] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:23.243383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.243752 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.243984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.274236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:23.276210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:23.289742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1195/2382] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:23.295720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.308153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109aaaa0-3e87-443a-806e-c5636afede53 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:23.313685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:23.314587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.315110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.316005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.316845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:23.317351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:23.320275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.336791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1188/2383] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:58:23.340873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.355007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8743188-5387-4c39-8e2e-ea15a46fd696 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:23.361087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:23.361645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.362619 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.362894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.363560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:23.366773 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.407995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1196/2384] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:23 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:23.410728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.424236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aba93e-e74a-42d1-8ed9-07802103cd69 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:23.521957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:23.523017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.523535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.524031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.525422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:23.527576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.558934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1189/2385] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:23 2022] POST /identity/v3/users => generated 291 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 15:58:23.559358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.571349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861fdb51-72b2-4638-816e-e3cdc9d2e3e7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:23.576958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:23.577584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.577866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.578777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.579442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:23.581401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.597144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1197/2386] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:23.598715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.620988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484288b-dedc-4c0b-8c2f-3a08a845c595 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:23.635846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be19bb-0290-4323-9ecf-3983f3051fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:23.636476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be19bb-0290-4323-9ecf-3983f3051fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.636750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be19bb-0290-4323-9ecf-3983f3051fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.637037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be19bb-0290-4323-9ecf-3983f3051fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.637694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be19bb-0290-4323-9ecf-3983f3051fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:23.639377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09be19bb-0290-4323-9ecf-3983f3051fd8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.663174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1190/2387] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:23 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:23.667695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.681554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a5aa6-3372-425f-bff9-6e201ab35f8c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:23.688358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:23.688856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.689142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.689496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] PATH_INFO: `/v3/projects/e0fa130dded4449b88e5d696f57a6a1d/users/c1320dca5c1840f9af2418e705e8aaa1/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.708247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0fa130dded4449b88e5d696f57a6a1d, user_id=c1320dca5c1840f9af2418e705e8aaa1, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:23.711450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:23.736646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1198/2388] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:23 2022] PUT /identity/v3/projects/e0fa130dded4449b88e5d696f57a6a1d/users/c1320dca5c1840f9af2418e705e8aaa1/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:58:23.741768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48b400-97c3-40a1-81cb-d44d34ab0a1d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:23.742050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48b400-97c3-40a1-81cb-d44d34ab0a1d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:23.742979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa48b400-97c3-40a1-81cb-d44d34ab0a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:23.755969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:23.788985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fa48b400-97c3-40a1-81cb-d44d34ab0a1d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:23.790126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fa48b400-97c3-40a1-81cb-d44d34ab0a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:23.924146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1191/2389] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:23.996011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:23.997904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:24.126479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc6c84-34e0-483f-8d9e-2ebf9e2c13b0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:24.130853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:24.131556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:24.131831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:24.132109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:24.162059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:24.164505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:24.190116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1199/2390] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:24.552254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89fba-05de-4bc8-812d-e604b616bebc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:24.552779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89fba-05de-4bc8-812d-e604b616bebc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:24.552779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba89fba-05de-4bc8-812d-e604b616bebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:24.584192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3ba89fba-05de-4bc8-812d-e604b616bebc None None] MFA Rules not processed for user `2fcfbaf5bab14a4783ca912a222341c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:24.586628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3ba89fba-05de-4bc8-812d-e604b616bebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:24.674407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1192/2391] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:25.116501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:25.118158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:25.134627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10c36ab-bbaa-4448-91a8-1418c04f8df1 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:25.139673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:25.140659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:25.141139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:25.141781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] PATH_INFO: `/v3/users/c4d688a00e5449499f08e7ece22ce159` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:25.143228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4d688a00e5449499f08e7ece22ce159)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:25.145515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:25.263005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4d688a00e5449499f08e7ece22ce159'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:25.268093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:25.277603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4d688a00e5449499f08e7ece22ce159'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:25.296152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4d688a00e5449499f08e7ece22ce159'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:25.306876 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4d688a00e5449499f08e7ece22ce159'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:25.308394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4d688a00e5449499f08e7ece22ce159'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:25.332568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1200/2392] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:25 2022] DELETE /identity/v3/users/c4d688a00e5449499f08e7ece22ce159 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:25.337248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:25.337539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:25.337821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:25.378437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:25.379767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:25.490233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1193/2393] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:25.506347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:25.508013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:25.646510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c825961-1bbc-4d1a-892f-5778e45691b4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:25.656000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6d857-4da6-4ffd-8d0b-a0af39561a7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:25.657626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d857-4da6-4ffd-8d0b-a0af39561a7d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:25.658050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d857-4da6-4ffd-8d0b-a0af39561a7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:25.658433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6d857-4da6-4ffd-8d0b-a0af39561a7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:25.688921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6d857-4da6-4ffd-8d0b-a0af39561a7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:25.694855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6d857-4da6-4ffd-8d0b-a0af39561a7d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:25.720245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1201/2394] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:26.374314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:26.376371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:26.400880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b119c-dfc5-4166-bb1d-8f1b43da8b2f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:26.426539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:26.429928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:26.430405 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:26.432106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] PATH_INFO: `/v3/projects/5c469a04e1114ad7a1761786485b294e` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:26.436966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c469a04e1114ad7a1761786485b294e)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:26.440300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:26.529187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:26.534926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c469a04e1114ad7a1761786485b294e', 'request_id': 'req-b6f1c55b-f4c8-4738-af00-da04a6c45d95'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:26.545735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c469a04e1114ad7a1761786485b294e', 'request_id': 'req-b6f1c55b-f4c8-4738-af00-da04a6c45d95'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:26.552869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c469a04e1114ad7a1761786485b294e', 'request_id': 'req-b6f1c55b-f4c8-4738-af00-da04a6c45d95'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:26.570195 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] The token cache is being invalidate because project 5c469a04e1114ad7a1761786485b294e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:26.570552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:26.571278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:26.573345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1194/2395] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:26 2022] DELETE /identity/v3/projects/5c469a04e1114ad7a1761786485b294e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:30.256447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.256987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.256987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.287503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] MFA Rules not processed for user `c1320dca5c1840f9af2418e705e8aaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:30.288810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:30.382176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1202/2396] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:30.386793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:30.389899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:30.464594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f1c55b-f4c8-4738-af00-da04a6c45d95 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:30.470109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:30.470725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.471014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.471283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.471971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:30.473898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:30.548021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1195/2397] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:30 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:30.550395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:30.565606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51af07d-6187-41df-badf-6c4a7a06a591 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:30.641584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:30.641584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.641850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.642094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.642606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:30.644601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:30.654900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:30.676490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1203/2398] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:30 2022] POST /identity/v3/users => generated 299 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:30.680860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:30.696174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726eb2ab-11dd-4caa-a7d7-4d1ad7ca5978 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:30.702127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:30.702817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.703081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.703396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.704722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:30.707065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:30.754678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1196/2399] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:30.764792 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:30.781203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37710b9-0b0c-48cb-8c2f-ef330a4f03d8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:30.786294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:30.787144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.787329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.787612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] PATH_INFO: `/v3/projects/740835c140554fdb956029ce5712ed82/users/d08677ad552e4282b91803df17da1c97/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.820128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=740835c140554fdb956029ce5712ed82, user_id=d08677ad552e4282b91803df17da1c97, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:30.822986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:30.846845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1204/2400] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:30 2022] PUT /identity/v3/projects/740835c140554fdb956029ce5712ed82/users/d08677ad552e4282b91803df17da1c97/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:30.851419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:30.866210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5456ebc-07c4-4b16-ade4-39e96c121c98 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:30.944333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b2f83-54f2-430d-a3e3-1f78f1263498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:30.945921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b2f83-54f2-430d-a3e3-1f78f1263498 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.946871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b2f83-54f2-430d-a3e3-1f78f1263498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.947508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b2f83-54f2-430d-a3e3-1f78f1263498 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.949053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b2f83-54f2-430d-a3e3-1f78f1263498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:30.951702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2b2f83-54f2-430d-a3e3-1f78f1263498 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:30.969014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1197/2401] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:30 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:30.972482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:30.983336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa2d456-d1b5-4ec7-acb2-6db9c2fc7d6f admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:30.988754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:30.989428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:30.989772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:30.989972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] PATH_INFO: `/v3/projects/740835c140554fdb956029ce5712ed82/users/d08677ad552e4282b91803df17da1c97/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:30.992020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] RBAC: Authorizing `identity:create_grant(project_id=740835c140554fdb956029ce5712ed82, user_id=d08677ad552e4282b91803df17da1c97, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:30.994696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:31.022771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1205/2402] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:30 2022] PUT /identity/v3/projects/740835c140554fdb956029ce5712ed82/users/d08677ad552e4282b91803df17da1c97/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:38.484207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41febd9a-26d5-4763-aae3-38f4ed4b6fdb None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:38.485044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41febd9a-26d5-4763-aae3-38f4ed4b6fdb None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:38.485573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41febd9a-26d5-4763-aae3-38f4ed4b6fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:38.519256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-41febd9a-26d5-4763-aae3-38f4ed4b6fdb None None] MFA Rules not processed for user `d08677ad552e4282b91803df17da1c97`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:38.520637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-41febd9a-26d5-4763-aae3-38f4ed4b6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:38.583902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:38.634192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1198/2403] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:38.644353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:38.646284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:38.723621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa847b-67f3-4010-ac2d-6ac49752855d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:38.727909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:38.728175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:38.728454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:38.728726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:38.799374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:38.802551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:38.813444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1206/2404] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:38.976705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26252d-ab98-4886-8dd7-b58bf93360e8 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:38.976997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26252d-ab98-4886-8dd7-b58bf93360e8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:38.977327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26252d-ab98-4886-8dd7-b58bf93360e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:39.002689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-df26252d-ab98-4886-8dd7-b58bf93360e8 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:39.002689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df26252d-ab98-4886-8dd7-b58bf93360e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:39.074088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1199/2405] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:58:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:39.132685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:39.134612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:39.202009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:39.202009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:39.202009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:39.240731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] MFA Rules not processed for user `7dc84ece6eb84315b54defd89be8a95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:39.241968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:39.275143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc014a5-4f0f-4d36-a49a-011e8b97d177 service glance] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:39.300722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6829-6b3f-4c0d-a807-ac01fe62fea6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:39.300952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6829-6b3f-4c0d-a807-ac01fe62fea6 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:39.301204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6829-6b3f-4c0d-a807-ac01fe62fea6 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:39.301484 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6829-6b3f-4c0d-a807-ac01fe62fea6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:39.319113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc6829-6b3f-4c0d-a807-ac01fe62fea6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:39.320797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc6829-6b3f-4c0d-a807-ac01fe62fea6 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:39.324941 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:39.348361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1207/2406] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:58:39.349039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1200/2407] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:45.901088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49b82f-d20a-4298-9cf8-f99c1a8e3b2e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:45.901686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49b82f-d20a-4298-9cf8-f99c1a8e3b2e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:45.901686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b49b82f-d20a-4298-9cf8-f99c1a8e3b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:45.944358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0b49b82f-d20a-4298-9cf8-f99c1a8e3b2e None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:45.945121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0b49b82f-d20a-4298-9cf8-f99c1a8e3b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:45.974354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1208/2408] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:58:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:45.974933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:45.991696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9240676a-4466-4157-98a9-5c16777e9bf0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:45.998831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e6c9e-d7e0-4eb8-9404-5122fd46488f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:46.000124 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e6c9e-d7e0-4eb8-9404-5122fd46488f service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:46.000621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e6c9e-d7e0-4eb8-9404-5122fd46488f service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:46.001121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e6c9e-d7e0-4eb8-9404-5122fd46488f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:46.011762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e6c9e-d7e0-4eb8-9404-5122fd46488f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:46.014565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e6c9e-d7e0-4eb8-9404-5122fd46488f service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:46.034115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1201/2409] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 15:58:51.460284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:51.461350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:51.461888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:51.488034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] MFA Rules not processed for user `c1320dca5c1840f9af2418e705e8aaa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:51.489721 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:51.504096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1209/2410] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:52.758122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1682-95f8-46a0-9825-e3c01f02cc50 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:52.759116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1682-95f8-46a0-9825-e3c01f02cc50 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:52.759565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc1682-95f8-46a0-9825-e3c01f02cc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:52.788283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e9dc1682-95f8-46a0-9825-e3c01f02cc50 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:52.789901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e9dc1682-95f8-46a0-9825-e3c01f02cc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:52.800695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1202/2411] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:58:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:52.817179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:52.839108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a12f9f-d354-4387-a150-7d2d8ac9032b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:52.846181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:52.847188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:52.847649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:52.848120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:52.862104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:52.864916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:52.878532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1210/2412] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:58:53.704513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582cb3d-ae3d-4cde-b392-d80656ab0700 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:53.704881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582cb3d-ae3d-4cde-b392-d80656ab0700 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:53.705224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582cb3d-ae3d-4cde-b392-d80656ab0700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:53.728105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7582cb3d-ae3d-4cde-b392-d80656ab0700 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:53.729907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7582cb3d-ae3d-4cde-b392-d80656ab0700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:53.741657 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1203/2413] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 15:58:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:53.779090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:53.795521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb9ed94-3113-473c-849e-1cb271a8f06b service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:53.814192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:53.814766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:53.815207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:53.815492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:53.828977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:53.831689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:53.848859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1211/2414] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:58:54.229403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080360d-bca6-47a6-ae44-11cd7dc37801 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:54.229930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080360d-bca6-47a6-ae44-11cd7dc37801 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:54.230300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0080360d-bca6-47a6-ae44-11cd7dc37801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:54.284946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0080360d-bca6-47a6-ae44-11cd7dc37801 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:54.286390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0080360d-bca6-47a6-ae44-11cd7dc37801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:54.293703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1204/2415] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:54.308052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:54.319063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4f90ef-bf76-4ace-968b-e05f29ecaeda service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:54.324372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:54.328545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:54.328929 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:54.329328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:54.339384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:54.341738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:54.355101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1212/2416] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:58:54.796851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ca39-2f07-4091-9c07-6e6627764d4d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:54.797213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ca39-2f07-4091-9c07-6e6627764d4d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:54.797552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300ca39-2f07-4091-9c07-6e6627764d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:54.829452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7300ca39-2f07-4091-9c07-6e6627764d4d None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:54.830686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7300ca39-2f07-4091-9c07-6e6627764d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:54.851293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1205/2417] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 15:58:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:54.894300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:54.925253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c066e9-63fe-484f-9782-b154f4df2547 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:54.940266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:54.941562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:54.942257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:54.942967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:54.960351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:54.964938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:54.986655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1213/2418] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:58:57.227189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:57.227640 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:57.227900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:57.261444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:57.264086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:57.285601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1206/2419] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:58:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:57.298557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:57.312454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0338d-287f-4e42-bf46-f08b48d7bf1e service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:57.322759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a198a-9953-4e5d-b012-17c23cc4ac20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:57.323758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a198a-9953-4e5d-b012-17c23cc4ac20 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:57.324237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a198a-9953-4e5d-b012-17c23cc4ac20 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:57.324687 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a198a-9953-4e5d-b012-17c23cc4ac20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:57.335421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a198a-9953-4e5d-b012-17c23cc4ac20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:57.337174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317a198a-9953-4e5d-b012-17c23cc4ac20 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:57.347593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1214/2420] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:58:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:58:57.886373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:57.888366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:57.994490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:58.065298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e9473-c339-460e-8b8f-e016f848fea5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:58.072027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:58.072784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:58.073198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:58.073547 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] PATH_INFO: `/v3/users/055ae4012f414796a9a82e177ce43af7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:58.080913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] RBAC: Authorizing `identity:delete_user(user_id=055ae4012f414796a9a82e177ce43af7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:58.083120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:58.189673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055ae4012f414796a9a82e177ce43af7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:58.207083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055ae4012f414796a9a82e177ce43af7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:58.209486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055ae4012f414796a9a82e177ce43af7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:58.239206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055ae4012f414796a9a82e177ce43af7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:58.240994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055ae4012f414796a9a82e177ce43af7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:58.286979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1207/2421] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:58:57 2022] DELETE /identity/v3/users/055ae4012f414796a9a82e177ce43af7 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:58.292382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:58.292795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:58.293284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:58.329573 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:58.331971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:58.561022 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1215/2422] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:58.611027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:58.613519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:58.790613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e7ec2-7be2-49bc-b242-edbc2770c33c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:58.795104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdc4d22-1604-4c92-8faa-e450f189f5f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:58.795759 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc4d22-1604-4c92-8faa-e450f189f5f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:58.796042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc4d22-1604-4c92-8faa-e450f189f5f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:58.796315 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdc4d22-1604-4c92-8faa-e450f189f5f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:58.806580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc4d22-1604-4c92-8faa-e450f189f5f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:58.807990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdc4d22-1604-4c92-8faa-e450f189f5f7 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:58.820155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1208/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:58:59.062703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.065065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:59.087509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992bf060-62c7-4102-8ab7-1d979bb10d99 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.093787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.094780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.095229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.095745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] PATH_INFO: `/v3/projects/0998ce41664645e6b2825dc435c8a4c2` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.097353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0998ce41664645e6b2825dc435c8a4c2)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:59.099320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:59.184588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0998ce41664645e6b2825dc435c8a4c2', 'request_id': 'req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:59.205406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0998ce41664645e6b2825dc435c8a4c2', 'request_id': 'req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:59.214323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0998ce41664645e6b2825dc435c8a4c2', 'request_id': 'req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:59.240053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] The token cache is being invalidate because project 0998ce41664645e6b2825dc435c8a4c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:58:59.240425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:59.241141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-522c89f0-ad74-4f90-ac72-4ba795eb6c4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:58:59.243413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1216/2424] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:58:59 2022] DELETE /identity/v3/projects/0998ce41664645e6b2825dc435c8a4c2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:58:59.248069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.248947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.249453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.271509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:59.272947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:59.353903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:59.376453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1209/2425] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:58:59.377861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.378355 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.378467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.386349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:58:59.405643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:58:59.407443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:58:59.417050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1217/2426] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 15:58:59.426535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.438363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812d5f1-6a1c-4e56-a74c-f43dc5f63db4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.443264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.444049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.444330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.444622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.445319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:58:59.445655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:59.447263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:59.468948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.479354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07386c-6209-436c-aa03-3c1996cdfcba None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.480047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1210/2427] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 15:58:59.488707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.489407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.489693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.489966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.490635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:59.494371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:59.560243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.561251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1218/2428] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:58:59 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:59.573655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9066f80-68a8-45e5-8ddb-78b0e4690cd4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.665589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.666340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.666685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.667037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.669789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:59.674752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:59.708683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1211/2429] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:58:59 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:58:59.713333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.740793 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e9e68d-f6da-48a7-977e-35c133c1c823 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.749295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.750106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.750392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.750747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.751468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:59.753523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:59.822676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.822949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1219/2430] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:59.838462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17e856-0b3f-4105-87c4-00fb3056a94e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.852900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac65be-a8e9-4e75-9166-7103c8a7f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.853751 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac65be-a8e9-4e75-9166-7103c8a7f869 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.854186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac65be-a8e9-4e75-9166-7103c8a7f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.854577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac65be-a8e9-4e75-9166-7103c8a7f869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:58:59.866927 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac65be-a8e9-4e75-9166-7103c8a7f869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:58:59.873419 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac65be-a8e9-4e75-9166-7103c8a7f869 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:58:59.888788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1212/2431] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:58:59.898445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:58:59.921636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8febc3-7d3b-4b2b-8c5d-2058e3d80a6c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:58:59.939470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:58:59.940193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:58:59.940474 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:58:59.940748 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] PATH_INFO: `/v3/projects/8213db22feb041c196d02aaeac22d875/users/c2e9cc0ef8394696b127a32edd75e885/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:00.003655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8213db22feb041c196d02aaeac22d875, user_id=c2e9cc0ef8394696b127a32edd75e885, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:00.003655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:00.029516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1220/2432] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:58:59 2022] PUT /identity/v3/projects/8213db22feb041c196d02aaeac22d875/users/c2e9cc0ef8394696b127a32edd75e885/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:59:00.034457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368a45-65f0-4bab-a30c-5e8f5a38739b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:00.034739 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368a45-65f0-4bab-a30c-5e8f5a38739b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:00.035020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98368a45-65f0-4bab-a30c-5e8f5a38739b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:00.096068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-98368a45-65f0-4bab-a30c-5e8f5a38739b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:00.103079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-98368a45-65f0-4bab-a30c-5e8f5a38739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:00.250900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1213/2433] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:00.260511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:00.262320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:00.351855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f222d3-f7a6-44d0-a659-92dee28aae57 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:00.355636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:00.356310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:00.357381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:00.357834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:00.371215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:00.372744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:00.386316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1221/2434] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:59:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:59:01.692189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4e395-ccf5-4ee6-9097-5e7c5e455ef6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:01.692880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4e395-ccf5-4ee6-9097-5e7c5e455ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:01.692880 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4e395-ccf5-4ee6-9097-5e7c5e455ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:01.760817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-49a4e395-ccf5-4ee6-9097-5e7c5e455ef6 None None] MFA Rules not processed for user `c2e9cc0ef8394696b127a32edd75e885`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:01.762111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-49a4e395-ccf5-4ee6-9097-5e7c5e455ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:01.828007 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:01.894771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1214/2435] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:59:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:01.904465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:01.906364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:02.013927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b378ad6-999e-4a46-8ecc-aa08510ad84d service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:02.024131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a966bfe-68cd-4128-a67f-620736fbfb90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:02.024936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966bfe-68cd-4128-a67f-620736fbfb90 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:02.025326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966bfe-68cd-4128-a67f-620736fbfb90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:02.026320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a966bfe-68cd-4128-a67f-620736fbfb90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:02.038736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a966bfe-68cd-4128-a67f-620736fbfb90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:02.043577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a966bfe-68cd-4128-a67f-620736fbfb90 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:02.078640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1222/2436] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:59:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:59:04.772086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:04.772865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:04.772865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:04.806415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] MFA Rules not processed for user `c2e9cc0ef8394696b127a32edd75e885`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:04.808051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:04.823009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1215/2437] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:59:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:08.080970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:08.081545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:08.081545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:08.102655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:08.104511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:08.148720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:08.169109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1223/2438] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 15:59:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:08.194143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:08.195648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:08.285937 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b9940-1f48-48dc-a4b2-d02252082f24 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:08.291947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:08.292788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:08.293144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:08.294009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:08.303364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:08.304997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:08.315758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1216/2439] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:59:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 15:59:10.561195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:10.563801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:10.581357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2401e2-5e61-45da-a8e8-0281e758331a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:10.586741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:10.588176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:10.588563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:10.588806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] PATH_INFO: `/v3/users/72a393cb6b2343d697ec2e87d750c70a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:10.590218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a393cb6b2343d697ec2e87d750c70a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:10.592615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:10.646064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a393cb6b2343d697ec2e87d750c70a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:10.661260 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a393cb6b2343d697ec2e87d750c70a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:10.672866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a393cb6b2343d697ec2e87d750c70a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:10.673639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a393cb6b2343d697ec2e87d750c70a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:10.674349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6579cc50-309a-4c89-ac32-ec71fa646f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a393cb6b2343d697ec2e87d750c70a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:10.689761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1224/2440] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 15:59:10 2022] DELETE /identity/v3/users/72a393cb6b2343d697ec2e87d750c70a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:59:10.870587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:10.906281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173216f-e422-49c7-9cc6-e155cab60008 service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:10.993998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:10.994629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:10.994905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:10.995176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] PATH_INFO: `/v3/projects/4ebf52e3718140d1b2e4883f8d5464f4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:10.996330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ebf52e3718140d1b2e4883f8d5464f4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:10.998299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.050998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ebf52e3718140d1b2e4883f8d5464f4', 'request_id': 'req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:11.059925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ebf52e3718140d1b2e4883f8d5464f4', 'request_id': 'req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:11.066799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ebf52e3718140d1b2e4883f8d5464f4', 'request_id': 'req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:11.079492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] The token cache is being invalidate because project 4ebf52e3718140d1b2e4883f8d5464f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 15:59:11.079846 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:11.080511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-72a32347-ebd6-4e93-b1ca-dca74e07c0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 15:59:11.087354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.087691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.088000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.091298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1217/2441] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 15:59:10 2022] DELETE /identity/v3/projects/4ebf52e3718140d1b2e4883f8d5464f4 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 15:59:11.108818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:11.110047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:11.187671 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1225/2442] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:11.195938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.196231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.196515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.206247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:11.222685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:11.224025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:11.233912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1218/2443] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:11.237927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.248493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756e00b9-43e0-4caf-9b2a-84fbc0f9600a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.253251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.253946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.254301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.254649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.255380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 15:59:11.255795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.257473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.269956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1226/2444] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 15:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 15:59:11.274173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.283428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fcf83-817f-4816-a496-5755c92f3fd5 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.296034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.296673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.297092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.297368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.298034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.300130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.346099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1219/2445] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 15:59:11 2022] POST /identity/v3/projects => generated 365 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:59:11.350232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.362453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e00863-ead5-4f2e-a042-8a0f4c2c2f69 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.417979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:11.435139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.435990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.436467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.437526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.438475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.440507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.464105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1227/2446] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 15:59:11 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 15:59:11.469084 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.481703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d26c3-cec9-492d-b995-3600d80a2956 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.486531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.487001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.487279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.487559 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.488206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.489770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.502119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1220/2447] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 15:59:11.513181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.527605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73148934-e604-4761-98d4-4715fafa0008 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.536321 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ec574-fc4e-4bd3-a179-5ca6bb6225f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.537019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ec574-fc4e-4bd3-a179-5ca6bb6225f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.537259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ec574-fc4e-4bd3-a179-5ca6bb6225f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.537585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ec574-fc4e-4bd3-a179-5ca6bb6225f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.538444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ec574-fc4e-4bd3-a179-5ca6bb6225f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.540836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ec574-fc4e-4bd3-a179-5ca6bb6225f4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.552823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1228/2448] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 15:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 15:59:11.561678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.573753 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6789b-d2dc-4608-a4eb-96459d99c06f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.578498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.579086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.579368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.579711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] PATH_INFO: `/v3/projects/a3e84eb57ce84e1d93a6901b8af65230/users/effa809750c144b4bbdbf3fc341b52f0/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.594758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3e84eb57ce84e1d93a6901b8af65230, user_id=effa809750c144b4bbdbf3fc341b52f0, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.597440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.612257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1221/2449] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 15:59:11 2022] PUT /identity/v3/projects/a3e84eb57ce84e1d93a6901b8af65230/users/effa809750c144b4bbdbf3fc341b52f0/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 15:59:11.622066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ba72e-21cf-4b40-b0f9-f3a2ac8bae00 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.622589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ba72e-21cf-4b40-b0f9-f3a2ac8bae00 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.622693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6ba72e-21cf-4b40-b0f9-f3a2ac8bae00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.651574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5c6ba72e-21cf-4b40-b0f9-f3a2ac8bae00 None None] MFA Rules not processed for user `effa809750c144b4bbdbf3fc341b52f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:11.653291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5c6ba72e-21cf-4b40-b0f9-f3a2ac8bae00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:11.716138 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1229/2450] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 15:59:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:11.727622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:11.729296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:11.786061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97cfef-c2f2-4140-b694-2818d8efbb59 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:11.789691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741977a-678e-40a0-90f0-aaefb79515f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:11.790289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741977a-678e-40a0-90f0-aaefb79515f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:11.790568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741977a-678e-40a0-90f0-aaefb79515f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:11.790836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741977a-678e-40a0-90f0-aaefb79515f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:11.799827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741977a-678e-40a0-90f0-aaefb79515f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:11.801241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741977a-678e-40a0-90f0-aaefb79515f8 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:11.809816 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1222/2451] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:59:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:59:15.342006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a031ba4-b194-42ea-a3f4-b2200506378b None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:15.342638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a031ba4-b194-42ea-a3f4-b2200506378b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:15.344113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a031ba4-b194-42ea-a3f4-b2200506378b None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:15.346298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1230/2452] 10.222.0.17 () {54 vars in 961 bytes} [Wed Sep 7 15:59:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 15:59:15.351387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:15.351928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:15.352014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:15.380682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] MFA Rules not processed for user `cc95332f41224ecdb42604ac98262545`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:15.381915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:15.477833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1223/2453] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Sep 7 15:59:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:15.486178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a031ba4-b194-42ea-a3f4-b2200506378b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:15.499650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a031ba4-b194-42ea-a3f4-b2200506378b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:15.505585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:15.506200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:15.506479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:15.506755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:15.508678 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:15.521925 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:15.523532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a057e7de-bd6c-4649-8449-7e805c286066 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:15.537888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1231/2454] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:59:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:59:25.468543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:25.530200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:48.028078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:48.029478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:48.086518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e691dab-b589-46ea-927b-7ec2530377f0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:48.089938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:48.090533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:48.090807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:48.091083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:48.092808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:48.161591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:48.162506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:48.172764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1224/2455] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 15:59:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 15:59:58.167723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=50 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 15:59:58.641866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:58.642608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:58.642608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:58.662732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 15:59:58.663975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:58.719533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1232/2456] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 15:59:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 15:59:58.741703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 15:59:58.743142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 15:59:58.811342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ca7e14-abeb-4b04-be21-2a3aca33b3c2 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 15:59:58.816823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f10a22a-d391-4d9f-aedd-479c0750fbee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 15:59:58.817426 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a22a-d391-4d9f-aedd-479c0750fbee service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 15:59:58.817703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a22a-d391-4d9f-aedd-479c0750fbee service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 15:59:58.817976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f10a22a-d391-4d9f-aedd-479c0750fbee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 15:59:58.826131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a22a-d391-4d9f-aedd-479c0750fbee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 15:59:58.827792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f10a22a-d391-4d9f-aedd-479c0750fbee service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 15:59:58.835899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1225/2457] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 15:59:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 07 16:00:08.711965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=25 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:00:08.832599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=32 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:00:15.050536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:15.158710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc301a4-d786-4e01-a827-29c35658887c None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:15.159971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:15.160776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:15.161537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:15.161537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] PATH_INFO: `/v3/users/effa809750c144b4bbdbf3fc341b52f0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:15.162536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] RBAC: Authorizing `identity:delete_user(user_id=effa809750c144b4bbdbf3fc341b52f0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:15.166103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:15.230987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'effa809750c144b4bbdbf3fc341b52f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.240661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'effa809750c144b4bbdbf3fc341b52f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.263894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'effa809750c144b4bbdbf3fc341b52f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.264749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'effa809750c144b4bbdbf3fc341b52f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.266098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'effa809750c144b4bbdbf3fc341b52f0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.284984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1233/2458] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:00:15 2022] DELETE /identity/v3/users/effa809750c144b4bbdbf3fc341b52f0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:00:15.290764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:15.291518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:15.291518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:15.313171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:00:15.314409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:15.389040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1226/2459] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:00:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:00:15.399342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:15.401193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:15.475416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc22d2-ebad-482e-9490-c68dbbf0aaca admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:15.479372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1029bd2-34b4-4f5c-93a2-984a868825bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:15.480099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1029bd2-34b4-4f5c-93a2-984a868825bd service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:15.480450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1029bd2-34b4-4f5c-93a2-984a868825bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:15.480790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1029bd2-34b4-4f5c-93a2-984a868825bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:15.492416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1029bd2-34b4-4f5c-93a2-984a868825bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:15.493888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1029bd2-34b4-4f5c-93a2-984a868825bd service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:15.505915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1234/2460] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:00:15.816137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:15.817818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:15.837422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9279cdc-f369-4e83-903e-9e8441c58100 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:15.844487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:15.845137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:15.845418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:15.845692 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] PATH_INFO: `/v3/projects/a3e84eb57ce84e1d93a6901b8af65230` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:15.846827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3e84eb57ce84e1d93a6901b8af65230)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:15.848947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:15.915234 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3e84eb57ce84e1d93a6901b8af65230', 'request_id': 'req-9b25aed6-b674-45e9-a338-1be5c46d3a5e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.929669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3e84eb57ce84e1d93a6901b8af65230', 'request_id': 'req-9b25aed6-b674-45e9-a338-1be5c46d3a5e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.936043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3e84eb57ce84e1d93a6901b8af65230', 'request_id': 'req-9b25aed6-b674-45e9-a338-1be5c46d3a5e'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.956333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] The token cache is being invalidate because project a3e84eb57ce84e1d93a6901b8af65230 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:00:15.956729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.957408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-9b25aed6-b674-45e9-a338-1be5c46d3a5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:00:15.961040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1227/2461] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:00:15 2022] DELETE /identity/v3/projects/a3e84eb57ce84e1d93a6901b8af65230 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:00:22.970809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993564f1-eaae-4649-bb38-63a080742168 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:22.971371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993564f1-eaae-4649-bb38-63a080742168 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:22.971469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993564f1-eaae-4649-bb38-63a080742168 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:22.972950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1235/2462] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 16:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:00:22.979154 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f96036-589a-4a9a-81e1-8aa7131652f3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:22.979644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f96036-589a-4a9a-81e1-8aa7131652f3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:22.979749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f96036-589a-4a9a-81e1-8aa7131652f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:22.999299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-89f96036-589a-4a9a-81e1-8aa7131652f3 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:00:23.000566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-89f96036-589a-4a9a-81e1-8aa7131652f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:23.073991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1228/2463] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 16:00:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:00:23.085835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993564f1-eaae-4649-bb38-63a080742168 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:23.087912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-993564f1-eaae-4649-bb38-63a080742168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:23.115200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:00:23.148829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993564f1-eaae-4649-bb38-63a080742168 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:23.152554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63101f-dcec-46c3-978e-cbebfaa88fe6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:23.153153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63101f-dcec-46c3-978e-cbebfaa88fe6 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.153442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63101f-dcec-46c3-978e-cbebfaa88fe6 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.153714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63101f-dcec-46c3-978e-cbebfaa88fe6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.165041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63101f-dcec-46c3-978e-cbebfaa88fe6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:23.166462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63101f-dcec-46c3-978e-cbebfaa88fe6 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:23.177767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1236/2464] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:00:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:00:23.256057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce66c8-8681-4174-b1fc-1fe2c0f3a43e None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.256438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce66c8-8681-4174-b1fc-1fe2c0f3a43e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.256805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce66c8-8681-4174-b1fc-1fe2c0f3a43e None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.258435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1229/2465] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 16:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:00:23.263164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc1785-57f1-4132-a845-02dbde8bbff3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.263454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc1785-57f1-4132-a845-02dbde8bbff3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.263769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adc1785-57f1-4132-a845-02dbde8bbff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.285267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2adc1785-57f1-4132-a845-02dbde8bbff3 None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:00:23.286512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2adc1785-57f1-4132-a845-02dbde8bbff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:23.342578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1237/2466] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:00:23.354478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce66c8-8681-4174-b1fc-1fe2c0f3a43e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:23.356231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6dce66c8-8681-4174-b1fc-1fe2c0f3a43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:23.369802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce66c8-8681-4174-b1fc-1fe2c0f3a43e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:23.373960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560963b3-c7bc-4524-999f-d8d54c574a00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:23.374616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560963b3-c7bc-4524-999f-d8d54c574a00 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.374898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560963b3-c7bc-4524-999f-d8d54c574a00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.375178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560963b3-c7bc-4524-999f-d8d54c574a00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.382854 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560963b3-c7bc-4524-999f-d8d54c574a00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:23.384808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560963b3-c7bc-4524-999f-d8d54c574a00 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:23.392796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1230/2467] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:00:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:00:23.474405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dce1b-ef2e-44f0-9d30-0466657fd897 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.474692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dce1b-ef2e-44f0-9d30-0466657fd897 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.475247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89dce1b-ef2e-44f0-9d30-0466657fd897 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.476638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1238/2468] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 16:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:00:23.481589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc827b2-8ff4-4858-8549-92c6766ba9c6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.481875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc827b2-8ff4-4858-8549-92c6766ba9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.482257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc827b2-8ff4-4858-8549-92c6766ba9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.490509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:00:23.504352 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-fcc827b2-8ff4-4858-8549-92c6766ba9c6 None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:00:23.505548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-fcc827b2-8ff4-4858-8549-92c6766ba9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:23.514641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1231/2469] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 16:00:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:00:23.524817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dce1b-ef2e-44f0-9d30-0466657fd897 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:23.536360 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89dce1b-ef2e-44f0-9d30-0466657fd897 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:23.540049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:23.540639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:23.540913 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:23.541185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:23.550986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:23.552443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:23.563824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1239/2470] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:00:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:00:32.251014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:32.251513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:32.251639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:32.270806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:00:32.271977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:32.324842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1232/2471] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 16:00:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:00:32.337104 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:32.338521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:32.406491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a99bfa-0b1e-4d16-81ec-365bee700857 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:32.411476 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:32.412106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:32.412385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:32.412657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:32.420754 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:32.422349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:32.430601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1240/2472] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 16:00:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:00:42.318336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=27 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:00:42.427183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=77 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:00:51.612487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:51.621677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5530cc7-36a4-413f-9268-eb38abd8488b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:51.625245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:51.626345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:51.626620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:51.626888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:51.628629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:51.643355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:51.644804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:51.655565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1233/2473] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:00:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:00:52.838393 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:00:52.839807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:52.852376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91163df9-a825-4c61-a559-4cbe6814c2c3 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:00:52.855877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:00:52.856495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:00:52.856772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:00:52.857041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:00:52.858661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:00:52.871711 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:00:52.873132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47562546-f2cd-4fb8-b9ac-574897d50653 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:00:52.884201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1241/2474] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:00:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:01:01.650790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:01:02.879304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:01:04.942981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:01:04.951568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8422e4a-1dbe-470b-84a2-76151d918760 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:01:04.955717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:01:04.956311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:04.956600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:04.956875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:04.958719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:05.016912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:01:05.018332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:01:05.027246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1234/2475] 10.222.0.17 () {60 vars in 1396 bytes} [Wed Sep 7 16:01:04 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:01:15.023092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=28 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:01:15.356519 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:15.356999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:15.357095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:15.376847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] MFA Rules not processed for user `c2e9cc0ef8394696b127a32edd75e885`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:01:15.378151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:15.435415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1242/2476] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:01:15.667189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:01:15.675633 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0683f607-5245-4c4a-9249-751ee6a83f2a service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:01:15.679248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:01:15.679874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:15.680149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:15.680416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:15.682108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:15.692721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:01:15.694188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d858066d-38e7-4843-ac46-fe06001edb1b service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:01:15.704407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1235/2477] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:01:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:01:16.008835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:01:16.010257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:16.082072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d1cf4-4d58-48e4-a232-41e7ee836f1b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:01:16.087197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:01:16.087834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:16.088137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:16.088408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:16.090102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:16.102048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:01:16.103636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:01:16.113169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1243/2478] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:01:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:01:17.042034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:17.042458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:17.042587 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:17.062092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:01:17.063295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:17.071608 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1236/2479] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 16:01:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:01:17.097726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:01:17.110654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c233a-9212-432a-92c5-97c47cba37d8 service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:01:17.127658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:01:17.128285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:17.128563 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:17.128832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:17.138171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:01:17.139768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:01:17.148098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1244/2480] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 16:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:01:18.729605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:01:18.738869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac6f3b-e67c-454a-ad74-9a5da78a8038 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:01:18.742404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:01:18.742996 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:01:18.743303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:01:18.743600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:01:18.745316 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:01:18.755228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:01:18.756680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:01:18.764579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1237/2481] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:01:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:01:27.144747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=63 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:01:28.761035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=15 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:01.415192 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:01.416683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:01.432392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae18537-58c0-42d0-913f-c7bce8207743 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:01.439379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:01.440284 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:01.440523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:01.440950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:01.442841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:01.456023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:01.457665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:01.468408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1245/2482] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:02:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:02:08.683776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:08.685200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:08.700722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366034a-9b55-47b6-9074-c672f8280d6c service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:08.706812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:08.707408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:08.707721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:08.707994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:08.709735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:08.769425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:08.771011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:08.780815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1238/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:02:08 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:02:11.463881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:18.775964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=28 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:28.965551 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:28.976833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331a71e3-2576-4063-a19d-5141304396c4 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:28.982281 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:28.982955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:28.983290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:28.983604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:28.985875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:28.998112 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.001423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.011137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1246/2484] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:02:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:02:29.018465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.020582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:29.090981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fe5228-32f7-4955-b4f0-1062f759a6fb service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.100391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.101016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.101296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.101571 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] PATH_INFO: `/v3/users/c2e9cc0ef8394696b127a32edd75e885` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.102683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e9cc0ef8394696b127a32edd75e885)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.106232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.150999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e9cc0ef8394696b127a32edd75e885'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.158704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e9cc0ef8394696b127a32edd75e885'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.159353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e9cc0ef8394696b127a32edd75e885'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.173243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e9cc0ef8394696b127a32edd75e885'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.173960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-060cd569-5900-405d-892a-88cbee5cb551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e9cc0ef8394696b127a32edd75e885'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.186217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1239/2485] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:02:29 2022] DELETE /identity/v3/users/c2e9cc0ef8394696b127a32edd75e885 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:02:29.358099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.369764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5c85b-a7d9-414f-87f5-9484f8132e0e service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.434436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.435270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.435696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.435934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] PATH_INFO: `/v3/projects/8213db22feb041c196d02aaeac22d875` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.437091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8213db22feb041c196d02aaeac22d875)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.439174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.500693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8213db22feb041c196d02aaeac22d875', 'request_id': 'req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.507669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8213db22feb041c196d02aaeac22d875', 'request_id': 'req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.513921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8213db22feb041c196d02aaeac22d875', 'request_id': 'req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.525274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] The token cache is being invalidate because project 8213db22feb041c196d02aaeac22d875 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:02:29.525562 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.526209 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7e7c0af3-1fb3-4cd0-99c7-32ff59bfc41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:02:29.527728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1247/2486] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:02:29 2022] DELETE /identity/v3/projects/8213db22feb041c196d02aaeac22d875 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:02:29.533283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.533574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.533848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.552946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:29.554143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:29.611398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1240/2487] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:02:29.619242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.619567 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.619855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.639564 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:29.641337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:29.649590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1248/2488] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:02:29.653820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.664018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cafae-1950-4572-ac56-7e25af7c8eb2 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.668570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.669158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.669435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.669706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.670348 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:02:29.670651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.672333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.683629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1241/2489] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:02:29.687739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.696861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ea0fa-5358-4b6b-a1f5-2218b48764d8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.701501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.702097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.702379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.702653 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.703830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.705764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.734262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1249/2490] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:02:29 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:02:29.738563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.748796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d52d1-eb95-4990-b8ea-71f1977a876b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.752386 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:29.806444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.807042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.807318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.807613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.808241 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.810175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.832138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1242/2491] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:02:29 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:02:29.836597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.846858 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9f98d-5dcb-4efc-9ecf-6ed7fa933288 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.851705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.852300 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.852581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.852853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.853497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.855108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.865183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1250/2492] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:02:29.869157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.879594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae316b9-e2fa-419a-8ce2-c41ab5709e69 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.884153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f69b6d-d4b1-41a7-ab1c-41c6232134b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.884741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f69b6d-d4b1-41a7-ab1c-41c6232134b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.885019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f69b6d-d4b1-41a7-ab1c-41c6232134b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.885291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f69b6d-d4b1-41a7-ab1c-41c6232134b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.885921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f69b6d-d4b1-41a7-ab1c-41c6232134b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.887494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f69b6d-d4b1-41a7-ab1c-41c6232134b5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.899461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1243/2493] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:02:29.903469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:29.913523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bf92d-93f6-4f69-9ba7-d71e83547ce8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:29.918152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:29.918791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.919073 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.919350 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] PATH_INFO: `/v3/projects/6ef2c5a9809b4bee95a171cf6dca4e2b/users/d595f32b5f2d406b9d6ce62acec1456b/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.933536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ef2c5a9809b4bee95a171cf6dca4e2b, user_id=d595f32b5f2d406b9d6ce62acec1456b, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:29.936344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:29.949207 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1251/2494] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:02:29 2022] PUT /identity/v3/projects/6ef2c5a9809b4bee95a171cf6dca4e2b/users/d595f32b5f2d406b9d6ce62acec1456b/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:02:29.954175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4615-f23a-4b9a-83fd-328975bfea7e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:29.954447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4615-f23a-4b9a-83fd-328975bfea7e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:29.954713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa4615-f23a-4b9a-83fd-328975bfea7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:29.974466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-84fa4615-f23a-4b9a-83fd-328975bfea7e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:29.975725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-84fa4615-f23a-4b9a-83fd-328975bfea7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:30.035062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1244/2495] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:02:30.045577 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:30.047319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:30.062243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:30.109749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddd98c8-567a-412a-8e5a-0c92b7a1722b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:30.113365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:30.113965 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:30.114242 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:30.114509 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:30.123237 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:30.124632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:30.134301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1252/2496] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:02:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:02:34.116666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377deeda-070f-4247-af46-a5604f53e816 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.117168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377deeda-070f-4247-af46-a5604f53e816 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.117268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377deeda-070f-4247-af46-a5604f53e816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.150537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-377deeda-070f-4247-af46-a5604f53e816 None None] MFA Rules not processed for user `d595f32b5f2d406b9d6ce62acec1456b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:34.152143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-377deeda-070f-4247-af46-a5604f53e816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:34.240609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1245/2497] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:02:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:02:34.245307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:34.256135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513bd4a8-4b56-430b-a035-8fa3b4f5b7fe service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:34.260860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:34.261532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.261804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.262381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.263045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:34.265053 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:34.303528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1253/2498] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:02:34 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:02:34.307796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377deeda-070f-4247-af46-a5604f53e816 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:34.319265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377deeda-070f-4247-af46-a5604f53e816 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:34.332808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:34.401840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:34.402686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.403077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.403538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.404486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:34.411645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:34.433901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1246/2499] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:02:34 2022] POST /identity/v3/users => generated 295 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:02:34.438442 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:34.468471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d147dd-df53-4a5a-a558-e152aa550f0c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:34.475273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:34.475940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.476220 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.476756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.477414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:34.479041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:34.490726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1254/2500] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:02:34.495732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:34.510075 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6d451a-b1d9-4138-88da-9e386a018b5e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:34.519109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:34.520034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.520312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.520582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] PATH_INFO: `/v3/projects/3d5e01b220564c38aeb39c88ac7ea530/users/6c641d4b768041269181ae8b299fb10d/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.538196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5e01b220564c38aeb39c88ac7ea530, user_id=6c641d4b768041269181ae8b299fb10d, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:34.541066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:34.557345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1247/2501] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:02:34 2022] PUT /identity/v3/projects/3d5e01b220564c38aeb39c88ac7ea530/users/6c641d4b768041269181ae8b299fb10d/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:02:34.561468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:34.572931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e351f09d-7c5c-4eef-8055-6fc725012168 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:34.637374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8694ab7-8f6e-4974-b16b-2103d26c4c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:34.637993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8694ab7-8f6e-4974-b16b-2103d26c4c18 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.638272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8694ab7-8f6e-4974-b16b-2103d26c4c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.638543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8694ab7-8f6e-4974-b16b-2103d26c4c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.639183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8694ab7-8f6e-4974-b16b-2103d26c4c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:34.641602 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8694ab7-8f6e-4974-b16b-2103d26c4c18 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:34.655420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1255/2502] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:02:34.659989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:34.674099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c7ab9-7998-4262-b6e2-c83d2ec90097 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:34.683969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adb62f9-c88a-4bcc-862a-006a08be5e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:34.684933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb62f9-c88a-4bcc-862a-006a08be5e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:34.685391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb62f9-c88a-4bcc-862a-006a08be5e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:34.685823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adb62f9-c88a-4bcc-862a-006a08be5e83 admin admin] PATH_INFO: `/v3/projects/3d5e01b220564c38aeb39c88ac7ea530/users/6c641d4b768041269181ae8b299fb10d/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:34.688992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb62f9-c88a-4bcc-862a-006a08be5e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d5e01b220564c38aeb39c88ac7ea530, user_id=6c641d4b768041269181ae8b299fb10d, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:34.691682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adb62f9-c88a-4bcc-862a-006a08be5e83 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:34.703916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1248/2503] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:02:34 2022] PUT /identity/v3/projects/3d5e01b220564c38aeb39c88ac7ea530/users/6c641d4b768041269181ae8b299fb10d/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:02:38.358456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:38.358961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:38.359210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:38.393486 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] MFA Rules not processed for user `7dc84ece6eb84315b54defd89be8a95c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:38.394856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:38.471588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1256/2504] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:02:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:02:39.811620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:39.813569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:39.814079 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:39.857478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] MFA Rules not processed for user `6c641d4b768041269181ae8b299fb10d`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:02:39.861089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:39.971103 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1249/2505] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:02:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:02:39.987790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:39.990193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:40.069818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:40.163572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0cc7e9-7a80-4b18-af4f-6b11afff4599 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:40.181020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f62ee1-5e71-4b99-8940-d791c77c7fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:40.182400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62ee1-5e71-4b99-8940-d791c77c7fa2 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:40.182928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62ee1-5e71-4b99-8940-d791c77c7fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:40.183268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f62ee1-5e71-4b99-8940-d791c77c7fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:40.247236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f62ee1-5e71-4b99-8940-d791c77c7fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:40.248909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f62ee1-5e71-4b99-8940-d791c77c7fa2 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:40.259365 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1257/2506] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:02:39 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:02:46.659279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:02:46.673113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f3e023-5cfe-4078-b80c-77ce6ff7f6ef None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:02:46.679252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:02:46.684256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:02:46.684616 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:02:46.685044 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:02:46.686979 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:02:46.699817 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:02:46.701444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:02:46.712989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1250/2507] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:02:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:02:50.255087 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=45 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:02:56.708729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=77,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:00.834233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b39e04-7a40-4a9d-9a28-5ac19f0e23ab None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:00.834769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b39e04-7a40-4a9d-9a28-5ac19f0e23ab None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:00.834769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b39e04-7a40-4a9d-9a28-5ac19f0e23ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:00.855715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f0b39e04-7a40-4a9d-9a28-5ac19f0e23ab None None] MFA Rules not processed for user `d7e96593e2814e8b9f7c0f320aa17c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:00.856935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f0b39e04-7a40-4a9d-9a28-5ac19f0e23ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:01.039171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1258/2508] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:03:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:03:01.364411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:01.366213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:01.426843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d18eed-a957-4483-ae8c-677975c47710 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:01.430512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:01.431118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:01.431398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:01.431706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:01.433443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:01.445119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:01.446546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:01.456655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1251/2509] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:03:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:03:07.422460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:07.422986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:07.422986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:07.446233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] MFA Rules not processed for user `d595f32b5f2d406b9d6ce62acec1456b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:03:07.447514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:07.459155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1259/2510] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:03:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:03:11.452268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=26 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:12.801040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:12.802600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:12.819295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512ea515-381f-4003-a186-0107f9f36deb service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:12.824808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:12.825408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:12.825689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:12.825965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:12.828278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:12.841042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:12.842682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:12.853492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1252/2511] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:03:17.440881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=28 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:22.849118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:25.308976 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:25.318491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046b26d-ab50-4cac-96f0-94adb50f78a6 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:25.321986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:25.322583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:25.322862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:25.323132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:25.331789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:25.333205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:25.342828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1260/2512] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:03:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:03:35.338266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=6 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:03:41.866647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:03:41.877853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae4497d-7078-4ef4-9d8f-ddebaa37e688 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:03:41.883256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:03:41.883907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:03:41.884184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:03:41.884462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:03:41.886165 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:03:41.943901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:03:41.945487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:03:41.954186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1253/2513] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:03:41 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:03:51.950201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=28 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:11.278544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:11.296429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7b824-e5fc-42a1-9baf-f6228ba2b9ee service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:11.300475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:11.301067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:11.301346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:11.301614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:11.303324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:11.421863 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:11.423860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:11.442933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1261/2514] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:14.536849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.539394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:14.620659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.622407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:14.637179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae510b2-cf73-4ab9-aa84-150d4db353d6 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.642637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.643253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.643584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.646645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] PATH_INFO: `/v3/users/7dc84ece6eb84315b54defd89be8a95c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.646645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dc84ece6eb84315b54defd89be8a95c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.647041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.695888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dc84ece6eb84315b54defd89be8a95c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:14.703948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dc84ece6eb84315b54defd89be8a95c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:14.716441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dc84ece6eb84315b54defd89be8a95c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:14.717088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dc84ece6eb84315b54defd89be8a95c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:14.717805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dc84ece6eb84315b54defd89be8a95c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:14.726857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:14.733169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1262/2515] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:04:14 2022] DELETE /identity/v3/users/7dc84ece6eb84315b54defd89be8a95c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:14.741137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12fd5cb-d701-468c-baae-1e53814175f5 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.808255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.808906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.809178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.809453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.811224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:14.822990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.824623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab97bf-66f0-47e1-8e5e-04999819ae6a service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:14.834898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1254/2516] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:14.917020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:14.918503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:14.983860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fa7f-576a-4211-b577-6f36ab07fbc4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:14.988986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:14.989624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:14.989895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:14.990169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] PATH_INFO: `/v3/projects/20596aac52254a159269aa634116f080` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:14.991267 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20596aac52254a159269aa634116f080)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:14.993215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.037995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20596aac52254a159269aa634116f080', 'request_id': 'req-1e9db42d-e5cf-4122-b1ad-ece517281434'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:15.045185 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20596aac52254a159269aa634116f080', 'request_id': 'req-1e9db42d-e5cf-4122-b1ad-ece517281434'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:15.051871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20596aac52254a159269aa634116f080', 'request_id': 'req-1e9db42d-e5cf-4122-b1ad-ece517281434'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:15.062244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] The token cache is being invalidate because project 20596aac52254a159269aa634116f080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:04:15.062529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:15.063170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1e9db42d-e5cf-4122-b1ad-ece517281434 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:15.064894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1263/2517] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:04:14 2022] DELETE /identity/v3/projects/20596aac52254a159269aa634116f080 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:15.069682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.069974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.070280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.104276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:15.128017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:15.129235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:15.189100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1255/2518] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:04:15.196908 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141bed5-6575-4406-846e-990f883f614a None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.197196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141bed5-6575-4406-846e-990f883f614a None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.197473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3141bed5-6575-4406-846e-990f883f614a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.217188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-3141bed5-6575-4406-846e-990f883f614a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:15.218382 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3141bed5-6575-4406-846e-990f883f614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:15.226964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1264/2519] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:04:15.231098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.240821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4fef36-d404-4af9-a660-3a3435149176 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.245373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.245968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.246248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.246523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.247170 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:04:15.247477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.249099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.260889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1256/2520] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:04:15.264946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141bed5-6575-4406-846e-990f883f614a None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.274691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3141bed5-6575-4406-846e-990f883f614a None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.279319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.279958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.280239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.280508 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.281136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.283645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.309463 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1265/2521] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:04:15 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:04:15.313744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.323650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68178d0-ced1-4eb1-93fd-01143ebebfd0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.382402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.383209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.383493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.383801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.384436 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.386353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.413217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1257/2522] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:04:15 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:04:15.414037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.424668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd55229-8889-4023-8e50-9b12a010c06e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.429361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.429978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.430261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.430536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.431190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.432847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.443823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1266/2523] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:04:15.448059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.458712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbb663-180f-4b56-984a-60c7002cf81f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.464344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887467fe-4f7a-45b5-9f22-d2eb7ce9107d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.464950 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887467fe-4f7a-45b5-9f22-d2eb7ce9107d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.465231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887467fe-4f7a-45b5-9f22-d2eb7ce9107d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.465505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887467fe-4f7a-45b5-9f22-d2eb7ce9107d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.466147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887467fe-4f7a-45b5-9f22-d2eb7ce9107d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.467877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887467fe-4f7a-45b5-9f22-d2eb7ce9107d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.479219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1258/2524] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:04:15.483707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.494096 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680e8dcd-c040-468f-837a-eaf80be8cc54 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.498592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.499239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.499521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.499832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] PATH_INFO: `/v3/projects/ab1963a1a6684af58f5c240787788b60/users/2dab44e593194fe788216b029eecd10c/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.517182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1963a1a6684af58f5c240787788b60, user_id=2dab44e593194fe788216b029eecd10c, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.519934 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.533295 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1267/2525] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:04:15 2022] PUT /identity/v3/projects/ab1963a1a6684af58f5c240787788b60/users/2dab44e593194fe788216b029eecd10c/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:15.538262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.538746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.538864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.557328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:15.558550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:15.608423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:15.617588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1259/2526] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:04:15.627525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:15.629216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:15.656183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:15.708246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4a91b-41ff-4d27-a3fd-8cb14f1976e7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:15.711818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e6538c-b428-4d64-9f64-a8f090532d7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:15.712418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6538c-b428-4d64-9f64-a8f090532d7b service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:15.712694 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6538c-b428-4d64-9f64-a8f090532d7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:15.712966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e6538c-b428-4d64-9f64-a8f090532d7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:15.721512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6538c-b428-4d64-9f64-a8f090532d7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:15.722926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e6538c-b428-4d64-9f64-a8f090532d7b service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:15.732317 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1268/2527] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:18.620960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:18.622564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:18.739869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e230cf41-1311-4bcc-9d6a-d2e5df7bab1e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:18.746019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:18.746650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:18.746931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:18.747205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:18.749048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:18.815925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:18.817133 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:18.829157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1260/2528] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:20.568676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a93f1-1989-4be8-be1d-65c1fc1d2cad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.569179 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a93f1-1989-4be8-be1d-65c1fc1d2cad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.569279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a93f1-1989-4be8-be1d-65c1fc1d2cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.599749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-323a93f1-1989-4be8-be1d-65c1fc1d2cad None None] MFA Rules not processed for user `2dab44e593194fe788216b029eecd10c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:20.600992 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-323a93f1-1989-4be8-be1d-65c1fc1d2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:20.675011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1269/2529] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:04:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:04:20.690376 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:20.702297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff84b96-2bc5-41bc-9421-c6068f693d61 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:20.710106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:20.710813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:20.711176 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:20.711637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:20.721011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:20.722903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:20.735592 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1261/2530] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:22.271324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb00ac-f98d-4945-8b25-0e736febb2d4 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:22.271861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb00ac-f98d-4945-8b25-0e736febb2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:22.272126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb00ac-f98d-4945-8b25-0e736febb2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:22.292441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-43fb00ac-f98d-4945-8b25-0e736febb2d4 None None] MFA Rules not processed for user `2dab44e593194fe788216b029eecd10c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:22.293660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-43fb00ac-f98d-4945-8b25-0e736febb2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:22.301399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1270/2531] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:04:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:04:24.200227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:24.201683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:24.262235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:24.276588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc4ce1-ef2d-45ee-9543-155dcdafcd45 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:24.282264 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:24.283003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:24.283287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:24.283591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:24.285349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:24.297410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:24.298995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:24.308968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1262/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:29.278156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280032b-2769-499f-8596-2de3042f73bb None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.278632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280032b-2769-499f-8596-2de3042f73bb None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.278740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7280032b-2769-499f-8596-2de3042f73bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.307524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7280032b-2769-499f-8596-2de3042f73bb None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:04:29.308728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7280032b-2769-499f-8596-2de3042f73bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:29.365990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1271/2533] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 16:04:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:04:29.417171 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.418638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:29.488121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca7c693-1c7d-418d-9cc6-5ade89da2493 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.493138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.493741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.494019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.494293 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.502823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.504459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.512642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1263/2534] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:04:29.872598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280032b-2769-499f-8596-2de3042f73bb None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:29.881897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7280032b-2769-499f-8596-2de3042f73bb None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:29.885495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:29.886082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:29.886374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:29.886644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:29.888381 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:29.900201 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:29.901652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:29.911691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1272/2535] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:04:36.804520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:36.806125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:36.819801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c70b70-c2d7-47c6-9b0d-cf788538a756 service swift] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:36.824407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:36.825025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:36.825301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:36.825569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] PATH_INFO: `/v3/users/c1320dca5c1840f9af2418e705e8aaa1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:36.826664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1320dca5c1840f9af2418e705e8aaa1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:36.829155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:36.874416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1320dca5c1840f9af2418e705e8aaa1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:36.881843 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1320dca5c1840f9af2418e705e8aaa1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:36.882460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1320dca5c1840f9af2418e705e8aaa1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:36.894848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1320dca5c1840f9af2418e705e8aaa1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:36.895527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1320dca5c1840f9af2418e705e8aaa1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:36.908424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1264/2536] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:04:36 2022] DELETE /identity/v3/users/c1320dca5c1840f9af2418e705e8aaa1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:37.076238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.087811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ac4f0-c8da-48b1-8bcf-0b0f9b452eca service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.098033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.148009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.148654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.148938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.149241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] PATH_INFO: `/v3/projects/e0fa130dded4449b88e5d696f57a6a1d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.150376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0fa130dded4449b88e5d696f57a6a1d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.152323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.215438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0fa130dded4449b88e5d696f57a6a1d', 'request_id': 'req-faa4f431-b6c2-4425-974e-bac646e91e6c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.222722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0fa130dded4449b88e5d696f57a6a1d', 'request_id': 'req-faa4f431-b6c2-4425-974e-bac646e91e6c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.229224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0fa130dded4449b88e5d696f57a6a1d', 'request_id': 'req-faa4f431-b6c2-4425-974e-bac646e91e6c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.239410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] The token cache is being invalidate because project e0fa130dded4449b88e5d696f57a6a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:04:37.239728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.240361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.241818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1273/2537] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:04:37 2022] DELETE /identity/v3/projects/e0fa130dded4449b88e5d696f57a6a1d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:37.245831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.247243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:37.260876 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c8d88-2b37-4726-b1cc-a633ef0986f6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.304953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.318400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.319023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.319303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.319604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] PATH_INFO: `/v3/users/d08677ad552e4282b91803df17da1c97` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.320735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d08677ad552e4282b91803df17da1c97)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.322673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.368104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd08677ad552e4282b91803df17da1c97'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.375507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd08677ad552e4282b91803df17da1c97'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.376147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd08677ad552e4282b91803df17da1c97'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.387803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd08677ad552e4282b91803df17da1c97'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.388422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-55af881c-7fdf-4abb-bebf-686686abc43a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd08677ad552e4282b91803df17da1c97'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.401098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1265/2538] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:04:37 2022] DELETE /identity/v3/users/d08677ad552e4282b91803df17da1c97 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:37.580889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:04:37.582398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:04:37.595383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa4f431-b6c2-4425-974e-bac646e91e6c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:04:37.658693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:04:37.659790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:04:37.660269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:04:37.660751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] PATH_INFO: `/v3/projects/740835c140554fdb956029ce5712ed82` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:04:37.663490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=740835c140554fdb956029ce5712ed82)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:04:37.665405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:04:37.715514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:37.719764 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740835c140554fdb956029ce5712ed82', 'request_id': 'req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.728513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740835c140554fdb956029ce5712ed82', 'request_id': 'req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.735945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740835c140554fdb956029ce5712ed82', 'request_id': 'req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.749081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] The token cache is being invalidate because project 740835c140554fdb956029ce5712ed82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:04:37.749371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.750548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:04:37.752063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1274/2539] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:04:37 2022] DELETE /identity/v3/projects/740835c140554fdb956029ce5712ed82 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:04:47.397554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=31,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:04:47.745975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:09.190993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f300dd-b959-4adf-bd00-f0d0494e4b75 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.191510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f300dd-b959-4adf-bd00-f0d0494e4b75 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.191510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f300dd-b959-4adf-bd00-f0d0494e4b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.211096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d1f300dd-b959-4adf-bd00-f0d0494e4b75 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:09.212324 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d1f300dd-b959-4adf-bd00-f0d0494e4b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:09.270299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1266/2540] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 16:05:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:05:09.283592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:09.285126 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:09.359713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a541a56-eaba-4b7d-8b3f-219d95cc6dbc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:09.364845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:09.365423 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:09.365682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:09.365955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:09.378045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:09.379698 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:09.388228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1275/2541] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 16:05:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:05:13.487939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:13.490035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:13.490522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:13.518865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] MFA Rules not processed for user `2fcfbaf5bab14a4783ca912a222341c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:13.520500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:13.608169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1267/2542] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:05:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:05:19.384732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=32 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:23.600464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=50 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:26.463314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.464904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.525189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853b747-ec12-418b-858e-c3fc1b8c8b7f service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:26.528758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:26.529495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.529640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.529912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.531780 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.621264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:26.622807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:26.634779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1276/2543] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:05:26.708564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.710060 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.769407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b698f71-9794-4a36-a26d-cc606797c65a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:26.772900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:26.773494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.773769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.774036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.775777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.785253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:26.786647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23f488-5a4a-453b-86ab-d6c7a402e18a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:26.794186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1268/2544] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:05:26.866357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:26.876409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f2151e-eb47-46a3-a838-1683ab10d351 service glance] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:26.879849 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:26.880443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:26.880722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:26.880989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:26.882697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:26.895755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:26.897156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:26.908348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1277/2545] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:05:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:05:29.992286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e938-9b15-4326-866c-f52c1580a515 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:29.992798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e938-9b15-4326-866c-f52c1580a515 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:29.992798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc0e938-9b15-4326-866c-f52c1580a515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:30.012185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ebc0e938-9b15-4326-866c-f52c1580a515 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:30.013432 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ebc0e938-9b15-4326-866c-f52c1580a515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:30.023997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1269/2546] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Sep 7 16:05:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:05:30.046499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:30.061514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112f0e2-2d0b-44e0-aef7-81dd680df41a service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:30.067438 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:30.067991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:30.068343 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:30.068692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:30.079084 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:30.081017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:30.091795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1278/2547] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 16:05:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:05:40.006434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=29 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:40.087245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=66 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:05:51.449935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:51.449935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:51.450377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:51.470014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] MFA Rules not processed for user `4411226c5c7c49d987880e64e363a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:51.471226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:51.478612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1270/2548] 10.222.0.17 () {58 vars in 967 bytes} [Wed Sep 7 16:05:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:05:51.491027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:51.501894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715f3648-2069-421b-b9c0-7110a81d8658 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:51.506964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f92a1a83dc5a4ababdebb85429de82b0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:51.507587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:51.507875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:51.508150 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:51.516896 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:51.518465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:51.526454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1279/2549] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Sep 7 16:05:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 07 16:05:53.085135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:53.094561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e372f-30e8-4ede-ace5-5825bbf75a90 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:53.098394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:53.098985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:53.099266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:53.099563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:53.101302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:53.114552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:53.116012 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a4b40-0ea3-4c3f-b098-7eaca084a006 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:53.127361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1271/2550] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:05:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:05:55.840666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:05:55.842108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:55.854665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ed97b-f17e-4f95-b979-88d45d391151 service swift] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:05:55.858174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:05:55.858765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:55.859041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:55.859314 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:55.860949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:55.873999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:05:55.875422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa4c5d-c94c-4743-b579-74aea11c3fe6 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:05:55.886434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1280/2551] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:05:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:05:59.442476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:05:59.442921 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:05:59.443035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:05:59.463494 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] MFA Rules not processed for user `d595f32b5f2d406b9d6ce62acec1456b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:05:59.464727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:05:59.518063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1272/2552] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:05:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:06:05.881768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=12 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:09.511035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=34 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:20.977402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:20.977936 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:20.978046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:20.997665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] MFA Rules not processed for user `2dab44e593194fe788216b029eecd10c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:06:20.998860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:21.056859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1281/2553] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:06:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:06:21.415225 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:21.426857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a7543-3b9a-4513-8aa6-753857400685 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:21.430456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:21.431273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:21.431764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:21.432219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:21.434169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:21.444464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:21.445935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:21.454655 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1273/2554] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:06:21.936078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:06:21.938427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:22.027346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1645896-492d-42ee-8feb-f204fbd28294 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:06:22.033413 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:06:22.034056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:06:22.034692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:06:22.035001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:06:22.036822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:06:22.049550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:06:22.051225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:06:22.061880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1282/2555] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:06:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:06:31.450805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:06:32.057993 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=57 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:00.296499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:00.297918 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:00.373303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058b994-1acd-4f49-a778-86cff0cc0ebe service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:00.376888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:00.377509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:00.377789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:00.378064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:00.379811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:00.457365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:00.458789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:00.477397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1274/2556] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:00 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:07.332672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:07.341957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a077a2-0ce0-4179-aa43-1b2853cc825c service placement] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:07.345604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:07.346205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:07.346489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:07.346763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:07.348768 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:07.412216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:07.413706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:07.425139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1283/2557] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:10.327904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:10.329398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:10.345374 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cde7c9-2d00-4826-b424-b63beb0b5975 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:10.350464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:10.351177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:10.351471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:10.351659 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] PATH_INFO: `/v3/users/d7e96593e2814e8b9f7c0f320aa17c3c` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:10.352805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7e96593e2814e8b9f7c0f320aa17c3c)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:10.354768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:10.409975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7e96593e2814e8b9f7c0f320aa17c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.419251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7e96593e2814e8b9f7c0f320aa17c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.420027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7e96593e2814e8b9f7c0f320aa17c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.429222 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:10.440599 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7e96593e2814e8b9f7c0f320aa17c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.441634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-cdd21819-968e-4c97-acd3-3d40d3562829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7e96593e2814e8b9f7c0f320aa17c3c'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.463815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1275/2558] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:10 2022] DELETE /identity/v3/users/d7e96593e2814e8b9f7c0f320aa17c3c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:10.647502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:10.662607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e60ea-f01e-4992-957a-4c73003def66 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:10.740175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:10.740812 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:10.741279 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:10.741565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] PATH_INFO: `/v3/projects/64aae5509a124aebbb92ff5dc82e90cf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:10.743116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] RBAC: Authorizing `identity:delete_project(project_id=64aae5509a124aebbb92ff5dc82e90cf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:10.745747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:10.796142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64aae5509a124aebbb92ff5dc82e90cf', 'request_id': 'req-6062c3d6-5f54-455a-bd14-c428e349a02f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.803683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64aae5509a124aebbb92ff5dc82e90cf', 'request_id': 'req-6062c3d6-5f54-455a-bd14-c428e349a02f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.810223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64aae5509a124aebbb92ff5dc82e90cf', 'request_id': 'req-6062c3d6-5f54-455a-bd14-c428e349a02f'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.820893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] The token cache is being invalidate because project 64aae5509a124aebbb92ff5dc82e90cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:10.821184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.821832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6062c3d6-5f54-455a-bd14-c428e349a02f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:10.823307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1284/2559] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:10 2022] DELETE /identity/v3/projects/64aae5509a124aebbb92ff5dc82e90cf => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:10.828875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:10.829169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:10.829445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:10.850297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:10.851493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:10.917227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1276/2560] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:10.921783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:10.922077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:10.922432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:10.945699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:10.946901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:10.955472 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1285/2561] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:10.960020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:10.970334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96357b10-fc0a-4883-a337-7c75eff3e9c4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:10.976036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:10.976642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:10.976941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:10.977322 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:10.977971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:10.978279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:10.979894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:10.995210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1277/2562] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:11.000675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:11.012415 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a648b1-05eb-4bf2-bfa3-4918bfca1bd7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:11.018496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:11.019134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.019410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.019713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.020340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:11.022236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:11.039856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:11.063306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1286/2563] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:10 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:11.069300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:11.079173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617be874-23da-4ffd-b555-ef8e17e72c69 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:11.137203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:11.137806 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.138088 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.138365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.138991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:11.140944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:11.162512 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1278/2564] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:11 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:11.166741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:11.177330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4d15c-f32f-443d-8683-6e8122b6dcae admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:11.182115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:11.182749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.183029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.183324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.184134 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:11.185724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:11.196214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1287/2565] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:11.200242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:11.210899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad357da7-5e83-4f25-9345-6989ac5df13a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:11.215558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101ed13-a2b2-48ab-a0a1-0aa170222bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:11.216162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ed13-a2b2-48ab-a0a1-0aa170222bae admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.216445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ed13-a2b2-48ab-a0a1-0aa170222bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.216720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101ed13-a2b2-48ab-a0a1-0aa170222bae admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.217361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101ed13-a2b2-48ab-a0a1-0aa170222bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:11.218975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101ed13-a2b2-48ab-a0a1-0aa170222bae admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:11.229903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1279/2566] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:11.233833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:11.243833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24115a-b495-484b-bd12-dc5c3d31695a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:11.248379 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:11.249046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.249332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.249607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] PATH_INFO: `/v3/projects/6441985a923843ccac3772964ecee145/users/e1de51f813d84d42bff895449042f195/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.264639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6441985a923843ccac3772964ecee145, user_id=e1de51f813d84d42bff895449042f195, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:11.267312 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:11.280815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1288/2567] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:11 2022] PUT /identity/v3/projects/6441985a923843ccac3772964ecee145/users/e1de51f813d84d42bff895449042f195/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:11.291163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.291456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.291768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.319335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] MFA Rules not processed for user `e1de51f813d84d42bff895449042f195`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:11.320613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:11.355186 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:11.388648 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1280/2568] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:11.400074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:11.401880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:11.457621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90612d75-68e8-4ef2-b463-dc8ba18d3795 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:11.461232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:11.461830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:11.462115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:11.462390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:11.470531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:11.471998 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:11.480959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1289/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:20.080669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:20.149390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b661ce-ee09-47df-b718-183967d9e205 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:20.150009 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:20.151387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.156972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:20.156972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.156972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.156972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] PATH_INFO: `/v3/users/e1de51f813d84d42bff895449042f195` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.157401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1de51f813d84d42bff895449042f195)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:20.158778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:20.206762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1de51f813d84d42bff895449042f195'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.213049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46399a1c-3c9f-460e-b824-f2d72b24ae60 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:20.215515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1de51f813d84d42bff895449042f195'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.216158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1de51f813d84d42bff895449042f195'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.216986 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:20.217588 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.217880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.218404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.219891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.228913 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1de51f813d84d42bff895449042f195'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.229666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-af66bf4e-593d-416e-a67c-101f0eac8cd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1de51f813d84d42bff895449042f195'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.232352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:20.233772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:20.243344 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1281/2570] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:20 2022] DELETE /identity/v3/users/e1de51f813d84d42bff895449042f195 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:20.246041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1290/2571] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:20.248160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.248453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.248731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.269695 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:20.271744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.320691 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:20.336284 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1282/2572] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:20.346599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:20.349045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.359970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09461244-bb37-455b-b50c-9cf36f246485 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:20.409624 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0568ea59-982d-4eb1-bd2a-a2cfb40dddd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:20.410243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568ea59-982d-4eb1-bd2a-a2cfb40dddd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.410526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568ea59-982d-4eb1-bd2a-a2cfb40dddd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.411333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0568ea59-982d-4eb1-bd2a-a2cfb40dddd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.420854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568ea59-982d-4eb1-bd2a-a2cfb40dddd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:20.422278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0568ea59-982d-4eb1-bd2a-a2cfb40dddd8 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:20.428453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:20.433595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1291/2573] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:20.719367 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:20.721246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.747716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1e1bf-6d05-45d9-ae02-75707e957b1a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:20.752595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:20.753235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.753518 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.753800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] PATH_INFO: `/v3/projects/6441985a923843ccac3772964ecee145` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.755569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6441985a923843ccac3772964ecee145)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:20.757549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:20.804327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6441985a923843ccac3772964ecee145', 'request_id': 'req-c70542d7-95a4-4a2b-99f4-3004970f8f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.812375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6441985a923843ccac3772964ecee145', 'request_id': 'req-c70542d7-95a4-4a2b-99f4-3004970f8f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.818594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6441985a923843ccac3772964ecee145', 'request_id': 'req-c70542d7-95a4-4a2b-99f4-3004970f8f03'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.830202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] The token cache is being invalidate because project 6441985a923843ccac3772964ecee145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:20.830508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.831346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c70542d7-95a4-4a2b-99f4-3004970f8f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:20.832887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1283/2574] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:20 2022] DELETE /identity/v3/projects/6441985a923843ccac3772964ecee145 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:20.837696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.837989 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.838286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.858582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:20.859969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.920590 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1292/2575] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:20.928847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.929213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.929557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.949401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:20.950723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:20.958799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1284/2576] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:20.962991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:20.973656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8ba46-88db-4b5c-9407-ad9067c700ce None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:20.978338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:20.978948 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:20.979231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:20.979506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:20.981938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:20.982253 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:20.983879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:20.996232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1293/2577] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:21.000489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.010265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8c2a3d-a4fb-4c5f-b6c4-6c8a57e55554 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.015500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.016145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.016427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.016700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.017336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.019235 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.048002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1285/2578] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:20 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:21.052811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.062912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6de49-359d-44ff-92f7-23119f73be29 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.120479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.121052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.121335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.121606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.122225 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.124176 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.147326 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1294/2579] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:21 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:21.151326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.162533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc5243-c803-49fb-80ce-3d2df4e04da0 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.167246 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.167900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.168181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.168452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.169100 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.170697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.188163 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1286/2580] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:21.189831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.200586 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f837d03-975c-4d30-92ca-9a8fc9bf232d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.205437 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34df158-bd2d-4405-87dc-1e5eecb9c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.206258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34df158-bd2d-4405-87dc-1e5eecb9c9ba admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.206543 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34df158-bd2d-4405-87dc-1e5eecb9c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.206817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34df158-bd2d-4405-87dc-1e5eecb9c9ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.208023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34df158-bd2d-4405-87dc-1e5eecb9c9ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.209645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34df158-bd2d-4405-87dc-1e5eecb9c9ba admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.221823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1295/2581] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:21.226255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.237936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012bc8a-a2b8-451d-997b-2e9b6a6ae2d3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.244356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.245026 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.245305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.245589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] PATH_INFO: `/v3/projects/641701cf4d8d4341a409801ca3af0d62/users/c1f83658f1794e8a8341bf0c824191d1/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.264536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=641701cf4d8d4341a409801ca3af0d62, user_id=c1f83658f1794e8a8341bf0c824191d1, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.267206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.285105 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1287/2582] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:21 2022] PUT /identity/v3/projects/641701cf4d8d4341a409801ca3af0d62/users/c1f83658f1794e8a8341bf0c824191d1/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:21.297626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.297918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.298205 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.333435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] MFA Rules not processed for user `c1f83658f1794e8a8341bf0c824191d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:21.334785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:21.398843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:21.411388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1296/2583] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:21.416620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.483850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100096b5-e5e4-45ec-9b24-d95b64aadf58 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.488612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.489707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.489987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.490287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.490912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.492852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.523355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:21.535609 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1288/2584] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:21 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:21.538109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.549195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9263900-a7a4-490e-abe6-3ecb1305dcd1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.616584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.617215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.617497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.617771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.618597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.620620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.643878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1297/2585] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:21 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:21.648427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.658801 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa1ce71-b82c-4183-bf29-539aec4b73df admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.663485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.664126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.664409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.664684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.665362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.666965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.678191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1289/2586] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:21.682710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.693427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea1c94-860d-47e7-b3b5-6a4c9ea06f30 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.698091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.698741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.699028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.699302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] PATH_INFO: `/v3/projects/ca6dcb15434048649e05a9b7b8e9e4dc/users/42df382be9834860b6cdd9b135697823/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.715405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6dcb15434048649e05a9b7b8e9e4dc, user_id=42df382be9834860b6cdd9b135697823, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.718222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.740747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1298/2587] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:21 2022] PUT /identity/v3/projects/ca6dcb15434048649e05a9b7b8e9e4dc/users/42df382be9834860b6cdd9b135697823/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:21.744465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.754810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21982c-1f79-4b17-ac0f-53c43a56e1fd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.820023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b261413-26d5-482d-9e7f-c511d1bba1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.820750 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261413-26d5-482d-9e7f-c511d1bba1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.821035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261413-26d5-482d-9e7f-c511d1bba1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.821310 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b261413-26d5-482d-9e7f-c511d1bba1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.822116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b261413-26d5-482d-9e7f-c511d1bba1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.823740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b261413-26d5-482d-9e7f-c511d1bba1e8 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.835683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1290/2588] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:21.839741 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:21.851059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b28461-0196-446f-88de-1006a2c037a7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:21.859041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:21.859818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.860168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.860531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] PATH_INFO: `/v3/projects/ca6dcb15434048649e05a9b7b8e9e4dc/users/42df382be9834860b6cdd9b135697823/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.862633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6dcb15434048649e05a9b7b8e9e4dc, user_id=42df382be9834860b6cdd9b135697823, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:21.865997 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:21.878410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1299/2589] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:21 2022] PUT /identity/v3/projects/ca6dcb15434048649e05a9b7b8e9e4dc/users/42df382be9834860b6cdd9b135697823/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:21.889144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:21.889429 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:21.889701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:21.920129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] MFA Rules not processed for user `42df382be9834860b6cdd9b135697823`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:21.921353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:22.004591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1291/2590] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:22.015861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:22.017595 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:22.083214 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01ed10-1eb3-48d9-b781-b889edb7c504 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:22.087180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa1caff-29d4-440e-b144-b4b5aac995ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:22.087872 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1caff-29d4-440e-b144-b4b5aac995ea service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:22.088157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1caff-29d4-440e-b144-b4b5aac995ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:22.088435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa1caff-29d4-440e-b144-b4b5aac995ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:22.098688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1caff-29d4-440e-b144-b4b5aac995ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:22.100198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa1caff-29d4-440e-b144-b4b5aac995ea service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:22.114842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1300/2591] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:22.548521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:22.618039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4a89c-f78d-4091-8450-ffd902a4aaa4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:22.623145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:22.623929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:22.624276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:22.624619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] PATH_INFO: `/v3/users/c1f83658f1794e8a8341bf0c824191d1` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:22.625877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f83658f1794e8a8341bf0c824191d1)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:22.627936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:22.649282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:22.695487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f83658f1794e8a8341bf0c824191d1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:22.703610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f83658f1794e8a8341bf0c824191d1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:22.706285 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f83658f1794e8a8341bf0c824191d1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:22.718175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f83658f1794e8a8341bf0c824191d1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:22.718904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f83658f1794e8a8341bf0c824191d1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:22.734427 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1292/2592] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:22 2022] DELETE /identity/v3/users/c1f83658f1794e8a8341bf0c824191d1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:22.740142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:22.740646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:22.740900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:22.764801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:22.766044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:22.817777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:22.833482 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1301/2593] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:22.843142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:22.844565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:22.901942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6415b0-9e88-4551-9739-5ddd98842c98 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:22.906054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:22.906668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:22.906948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:22.907218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:22.918108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:22.919556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:22.929658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1293/2594] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:23.257262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:23.258745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:23.280356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1509b6bc-5d29-41cf-a2a6-9d0765b57eaf None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:23.291183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:23.291880 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:23.292166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:23.292449 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] PATH_INFO: `/v3/projects/641701cf4d8d4341a409801ca3af0d62` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:23.294538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=641701cf4d8d4341a409801ca3af0d62)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:23.296631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:23.367979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641701cf4d8d4341a409801ca3af0d62', 'request_id': 'req-91608722-8076-41a7-993b-060eb4415d87'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.376841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641701cf4d8d4341a409801ca3af0d62', 'request_id': 'req-91608722-8076-41a7-993b-060eb4415d87'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.387007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641701cf4d8d4341a409801ca3af0d62', 'request_id': 'req-91608722-8076-41a7-993b-060eb4415d87'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.401331 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] The token cache is being invalidate because project 641701cf4d8d4341a409801ca3af0d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:23.401647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.402294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.407046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1302/2595] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:23 2022] DELETE /identity/v3/projects/641701cf4d8d4341a409801ca3af0d62 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:23.408272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:23.418005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64594bc0-cc82-45d7-bd1a-0e1183c063a5 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:23.501948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:23.503112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:23.503638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:23.504129 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] PATH_INFO: `/v3/users/42df382be9834860b6cdd9b135697823` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:23.507028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42df382be9834860b6cdd9b135697823)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:23.509214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:23.581226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:23.604444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42df382be9834860b6cdd9b135697823'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.618350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42df382be9834860b6cdd9b135697823'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.619680 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42df382be9834860b6cdd9b135697823'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.641663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42df382be9834860b6cdd9b135697823'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.643142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42df382be9834860b6cdd9b135697823'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.671487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1294/2596] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:23 2022] DELETE /identity/v3/users/42df382be9834860b6cdd9b135697823 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:23.693088 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:23.696433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:23.716254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91608722-8076-41a7-993b-060eb4415d87 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:23.777064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:23.777830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:23.778302 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:23.778677 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] PATH_INFO: `/v3/users/2fcfbaf5bab14a4783ca912a222341c0` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:23.781254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fcfbaf5bab14a4783ca912a222341c0)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:23.783337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:23.832078 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fcfbaf5bab14a4783ca912a222341c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.842024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fcfbaf5bab14a4783ca912a222341c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.855384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fcfbaf5bab14a4783ca912a222341c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.856142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fcfbaf5bab14a4783ca912a222341c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.856759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fcfbaf5bab14a4783ca912a222341c0'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:23.865623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:23.875493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1303/2597] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:23 2022] DELETE /identity/v3/users/2fcfbaf5bab14a4783ca912a222341c0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:23.961929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:23.963377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:23.978871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05d7ef-aa20-40b9-955e-646ce8a5c9d4 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.042647 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.043292 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.043607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.043884 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] PATH_INFO: `/v3/projects/ca6dcb15434048649e05a9b7b8e9e4dc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.045140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6dcb15434048649e05a9b7b8e9e4dc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.047193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.056218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.057686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.071056 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b609b3-e86e-4832-9212-0dbc79624a17 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.076521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.077154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.077434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.077862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] PATH_INFO: `/v3/projects/a446dbd47db84c4987a3ee5332356b84` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.079113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a446dbd47db84c4987a3ee5332356b84)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.081232 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.104777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6dcb15434048649e05a9b7b8e9e4dc', 'request_id': 'req-6a3c61d2-8065-4dae-8038-c69bdac2cb42'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.112911 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6dcb15434048649e05a9b7b8e9e4dc', 'request_id': 'req-6a3c61d2-8065-4dae-8038-c69bdac2cb42'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.118892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6dcb15434048649e05a9b7b8e9e4dc', 'request_id': 'req-6a3c61d2-8065-4dae-8038-c69bdac2cb42'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.131256 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] The token cache is being invalidate because project ca6dcb15434048649e05a9b7b8e9e4dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:24.131570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.132230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6a3c61d2-8065-4dae-8038-c69bdac2cb42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.132402 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a446dbd47db84c4987a3ee5332356b84', 'request_id': 'req-8688e076-9949-45bb-a2fd-69ac9a2f18c8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.137407 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1295/2598] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:23 2022] DELETE /identity/v3/projects/ca6dcb15434048649e05a9b7b8e9e4dc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:24.138782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96ad46-6663-4fc3-a891-8eda6d515b86 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.139251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96ad46-6663-4fc3-a891-8eda6d515b86 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.139448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96ad46-6663-4fc3-a891-8eda6d515b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.140760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a446dbd47db84c4987a3ee5332356b84', 'request_id': 'req-8688e076-9949-45bb-a2fd-69ac9a2f18c8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.148236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a446dbd47db84c4987a3ee5332356b84', 'request_id': 'req-8688e076-9949-45bb-a2fd-69ac9a2f18c8'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.158638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] The token cache is being invalidate because project a446dbd47db84c4987a3ee5332356b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:24.158924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.159634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-8688e076-9949-45bb-a2fd-69ac9a2f18c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:24.160358 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ed96ad46-6663-4fc3-a891-8eda6d515b86 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:24.161232 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ed96ad46-6663-4fc3-a891-8eda6d515b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.161778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1304/2599] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:24 2022] DELETE /identity/v3/projects/a446dbd47db84c4987a3ee5332356b84 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:24.166377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca844b-8186-49fe-a8d1-c244a8654623 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.166727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca844b-8186-49fe-a8d1-c244a8654623 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.167007 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca844b-8186-49fe-a8d1-c244a8654623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.186866 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b4ca844b-8186-49fe-a8d1-c244a8654623 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:24.188103 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4ca844b-8186-49fe-a8d1-c244a8654623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.199904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:24.227528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1296/2600] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:24.235335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.235654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.235935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.250660 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1305/2601] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:24.256245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:24.257456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.259149 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.259450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.259745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.265860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1297/2602] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:24.269933 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.278714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba7867c-b86a-40ea-aff2-e29f5b534f9b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.281210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:24.282409 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.283305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.283915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.284193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.284462 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.285095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:24.285393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.287400 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.292580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1306/2603] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:24.296324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.303520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1298/2604] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:07:24.306611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866237f4-8f3f-46f2-896b-4f6d2b016226 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.307825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.311301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.311972 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.312255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.312536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.313541 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:24.313856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.316075 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.318199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d71329-7d0d-48e3-85a7-dbd83f4c1b06 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.322732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.323307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.323610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.323886 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.324501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.326365 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.328348 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1307/2605] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:07:24.332635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.342110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c041706-0d79-413c-b7f6-944392230216 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.347011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.347904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.348222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.348503 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.349133 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.351061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.358172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1299/2606] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:24.358855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.381657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1308/2607] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:24.385926 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.423905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:24.424116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72b0a8-d08c-4a82-8e28-5ee63078ccfb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.449481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e37be5-5d9a-4d7d-b3d7-8307194b3ed8 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.454170 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.454777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.455065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.455341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.456023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.458502 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.481142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1309/2608] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:24.484716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.485325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.485459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.485800 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.486139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.486827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.488963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.494749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d2fe8-66e3-4a81-a23a-60cc5e499d0a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.499318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.499950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.500248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.500505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.501171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.503210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.509722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1300/2609] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:24.513737 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.513894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1310/2610] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:24.517556 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.524003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae0cc23-943e-4426-8230-71009ffcd168 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.527429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db55c2d-29ee-4d0a-bbc0-eccb890ec2e9 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.528602 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.529189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.529464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.529735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.530362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.532063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.532553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.533198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.533552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.533901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.534528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.536686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.542402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1301/2611] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:24.546139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.550265 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1311/2612] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:24.551231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.555509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2766bb88-3238-47dc-9fad-829977757c95 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.560615 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62371a-0307-4ff9-809a-841b5f2211f2 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.560763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.561190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.561506 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.561736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.562368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.563961 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.565979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43acb426-b1d8-4c2b-9664-f8dbd2d6af0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.566715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acb426-b1d8-4c2b-9664-f8dbd2d6af0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.567571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acb426-b1d8-4c2b-9664-f8dbd2d6af0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.568044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43acb426-b1d8-4c2b-9664-f8dbd2d6af0f admin admin] PATH_INFO: `/v3/projects/79bea044ec2f451495ece1ff12b2b010/users/f3b9a332fac8487b808aa681d3e9a5b2/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.575408 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1302/2613] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:24.579450 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.585083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acb426-b1d8-4c2b-9664-f8dbd2d6af0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=79bea044ec2f451495ece1ff12b2b010, user_id=f3b9a332fac8487b808aa681d3e9a5b2, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.587759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43acb426-b1d8-4c2b-9664-f8dbd2d6af0f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.589811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5868a44-4222-489d-bd8e-df8e962424ee admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.595787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a726c7-bc6c-4bec-ba9f-26249f881c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.596536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a726c7-bc6c-4bec-ba9f-26249f881c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.597489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a726c7-bc6c-4bec-ba9f-26249f881c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.597754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a726c7-bc6c-4bec-ba9f-26249f881c0c admin admin] PATH_INFO: `/v3/projects/14f76e0c775d4b1e8427cd4440a8bb1a/users/1f6f7bc702bb4e9592c38cae6d30b300/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.601824 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1312/2614] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:24 2022] PUT /identity/v3/projects/79bea044ec2f451495ece1ff12b2b010/users/f3b9a332fac8487b808aa681d3e9a5b2/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:24.609567 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:24.612545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.612813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.613100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.614002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a726c7-bc6c-4bec-ba9f-26249f881c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=14f76e0c775d4b1e8427cd4440a8bb1a, user_id=1f6f7bc702bb4e9592c38cae6d30b300, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.617444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a726c7-bc6c-4bec-ba9f-26249f881c0c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.631368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1303/2615] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:24 2022] PUT /identity/v3/projects/14f76e0c775d4b1e8427cd4440a8bb1a/users/1f6f7bc702bb4e9592c38cae6d30b300/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:24.642444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.642729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.643005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.647969 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] MFA Rules not processed for user `f3b9a332fac8487b808aa681d3e9a5b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:24.649213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.671694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] MFA Rules not processed for user `1f6f7bc702bb4e9592c38cae6d30b300`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:24.673370 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:24.715135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1313/2616] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:24.719299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.741167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1304/2617] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:24.745637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.782669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b42f4-2ebc-46dc-b320-f0850929951f None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.787974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.788581 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.788898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.789173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.789803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.791732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.806813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:24.808417 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dae2f6-e842-4744-9a4c-500a6f00ce63 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.814401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.815164 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.815440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.815749 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.816414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.818364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.820180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1314/2618] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:24.823914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.849840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1305/2619] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:24.850766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.885937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0306c7d5-77da-41ab-8526-2c812020e2a1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.912482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f95988-619a-4b2c-a62c-2204b8b7df63 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.917276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.917870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.918147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.918413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.919028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.920945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.931381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:24.943180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.943822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.944102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.944238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1306/2620] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:24.944362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.945062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.947089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.948143 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.958148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1f8d9-baaa-46b6-9f0a-64a4c872a27e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.963453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.964080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.964416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.964701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.965351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.966958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.971191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1315/2621] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:24 2022] POST /identity/v3/users => generated 300 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:24.972517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.978201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1307/2622] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:24.982422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:24.984616 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef2025-86a0-47ee-9e93-757d3a8ad5a4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.989308 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.989892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.990166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.990433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:24.991063 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:24.992111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6545c-db58-47bc-9d97-5d3c29f712a1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:24.992665 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:24.997097 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:24.998064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:24.998064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:24.998267 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] PATH_INFO: `/v3/projects/d6a4602ec81f4cb3a0ef193138bc3990/users/f314001cf4e24357bc0aecdf8c18dceb/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.002650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1316/2623] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:25.006571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.014821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a4602ec81f4cb3a0ef193138bc3990, user_id=f314001cf4e24357bc0aecdf8c18dceb, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.016050 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd19f4b-a536-4934-a719-a99ea23ec60a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.017685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.021200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.021958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.022184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.022456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] PATH_INFO: `/v3/projects/087c11aa849b46f89414139ed46fb748/users/fe80488e8f7f4b968eb526af83fb1cbb/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.033703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1308/2624] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:24 2022] PUT /identity/v3/projects/d6a4602ec81f4cb3a0ef193138bc3990/users/f314001cf4e24357bc0aecdf8c18dceb/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:25.034857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.038633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087c11aa849b46f89414139ed46fb748, user_id=fe80488e8f7f4b968eb526af83fb1cbb, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.041585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.054478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1317/2625] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:25 2022] PUT /identity/v3/projects/087c11aa849b46f89414139ed46fb748/users/fe80488e8f7f4b968eb526af83fb1cbb/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:25.058187 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.102898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15b678-8c4d-4c91-b52e-2ec757665d96 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.123613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:25.128638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9eba57-9f4c-473c-9944-bc9a71f34021 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.135384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.136106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.136395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.136672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.137375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.138978 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.149077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1318/2626] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:25.153390 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.162790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.163404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.163712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.163983 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.164623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.165873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181e44c7-bd30-4dd9-8898-b60e5f90e088 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.166226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.171090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04fdd52-b215-4256-88d7-2d0184e48970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.172219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fdd52-b215-4256-88d7-2d0184e48970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.172510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fdd52-b215-4256-88d7-2d0184e48970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.172789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04fdd52-b215-4256-88d7-2d0184e48970 admin admin] PATH_INFO: `/v3/projects/087c11aa849b46f89414139ed46fb748/users/fe80488e8f7f4b968eb526af83fb1cbb/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.174715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fdd52-b215-4256-88d7-2d0184e48970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=087c11aa849b46f89414139ed46fb748, user_id=fe80488e8f7f4b968eb526af83fb1cbb, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.177831 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1309/2627] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:25 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:25.178008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04fdd52-b215-4256-88d7-2d0184e48970 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.182115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.188759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1319/2628] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:25 2022] PUT /identity/v3/projects/087c11aa849b46f89414139ed46fb748/users/fe80488e8f7f4b968eb526af83fb1cbb/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:25.192847 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d427d-7988-4e33-bf2d-70bc3ae53f72 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.200892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.201182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.201459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.229264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] MFA Rules not processed for user `fe80488e8f7f4b968eb526af83fb1cbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:25.230522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:25.248040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172a1767-ce86-4615-b99e-8eb045071f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.248629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a1767-ce86-4615-b99e-8eb045071f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.248905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a1767-ce86-4615-b99e-8eb045071f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.249173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172a1767-ce86-4615-b99e-8eb045071f20 admin admin] PATH_INFO: `/v3/projects/d6a4602ec81f4cb3a0ef193138bc3990/users/f314001cf4e24357bc0aecdf8c18dceb/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.251078 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a1767-ce86-4615-b99e-8eb045071f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6a4602ec81f4cb3a0ef193138bc3990, user_id=f314001cf4e24357bc0aecdf8c18dceb, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.253724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172a1767-ce86-4615-b99e-8eb045071f20 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.257565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:25.267106 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1310/2629] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:25 2022] PUT /identity/v3/projects/d6a4602ec81f4cb3a0ef193138bc3990/users/f314001cf4e24357bc0aecdf8c18dceb/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:25.277041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.277451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.277736 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.303431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1320/2630] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:25.306002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] MFA Rules not processed for user `f314001cf4e24357bc0aecdf8c18dceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:25.307328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:25.314322 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.315744 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:25.375613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1311/2631] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:25.375837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c9b28-9eac-41ef-9899-5d450dca0ad7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.386172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.387576 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:25.425524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e91fdd-f463-41ec-8394-ac58c1676e3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.426813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91fdd-f463-41ec-8394-ac58c1676e3d service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.427004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91fdd-f463-41ec-8394-ac58c1676e3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.427274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e91fdd-f463-41ec-8394-ac58c1676e3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.444257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dae2dc-077d-40da-9d6d-4217cadbfd15 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.447962 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.448570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.448873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.449145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.459109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.460477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:25.460619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.470663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1312/2632] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:25.489641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e91fdd-f463-41ec-8394-ac58c1676e3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.491289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e91fdd-f463-41ec-8394-ac58c1676e3d service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.501492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1321/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:25.534482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.601434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f67b1-0919-4227-bb33-62bd1d479b10 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.606140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.606960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.607249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.607664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] PATH_INFO: `/v3/users/f3b9a332fac8487b808aa681d3e9a5b2` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.609122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3b9a332fac8487b808aa681d3e9a5b2)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.611566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.643702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:25.665557 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3b9a332fac8487b808aa681d3e9a5b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:25.677203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3b9a332fac8487b808aa681d3e9a5b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:25.677871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3b9a332fac8487b808aa681d3e9a5b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:25.691298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3b9a332fac8487b808aa681d3e9a5b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:25.692048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3b9a332fac8487b808aa681d3e9a5b2'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:25.706008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1313/2634] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:25 2022] DELETE /identity/v3/users/f3b9a332fac8487b808aa681d3e9a5b2 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:25.711229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.711520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.711963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.732028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:25.733277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:25.803724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1322/2635] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:25.813569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:25.815104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:25.873403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12e7ca-14f6-4f4a-b011-1a2205507ba3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:25.877202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdead5-5abf-4c06-a7f0-c1f9213e3e17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:25.877901 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdead5-5abf-4c06-a7f0-c1f9213e3e17 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:25.878251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdead5-5abf-4c06-a7f0-c1f9213e3e17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:25.878590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdead5-5abf-4c06-a7f0-c1f9213e3e17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:25.888248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdead5-5abf-4c06-a7f0-c1f9213e3e17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:25.889708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdead5-5abf-4c06-a7f0-c1f9213e3e17 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:25.899907 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1314/2636] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:26.067499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.068713 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.080490 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6051af87-2548-4bc8-a333-f891a11baeb9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:26.085235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:26.086116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.086389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.086662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] PATH_INFO: `/v3/users/1f6f7bc702bb4e9592c38cae6d30b300` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.088320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6f7bc702bb4e9592c38cae6d30b300)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:26.090243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:26.138910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6f7bc702bb4e9592c38cae6d30b300'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.146485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6f7bc702bb4e9592c38cae6d30b300'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.160271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6f7bc702bb4e9592c38cae6d30b300'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.161514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6f7bc702bb4e9592c38cae6d30b300'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.163089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6f7bc702bb4e9592c38cae6d30b300'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.175120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1323/2637] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:26 2022] DELETE /identity/v3/users/1f6f7bc702bb4e9592c38cae6d30b300 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:26.180266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.180527 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.180784 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.191064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.192807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.201109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:26.202660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.241428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:26.263461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bfc01-370e-45b0-8a4c-a0dd24303f0b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:26.268034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:26.268663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.268940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.269218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] PATH_INFO: `/v3/projects/79bea044ec2f451495ece1ff12b2b010` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.270357 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79bea044ec2f451495ece1ff12b2b010)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:26.272307 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:26.277437 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1315/2638] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:26.288131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.289489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.328099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79bea044ec2f451495ece1ff12b2b010', 'request_id': 'req-51036000-8ad0-440b-84de-e2353a0a7d78'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.338243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79bea044ec2f451495ece1ff12b2b010', 'request_id': 'req-51036000-8ad0-440b-84de-e2353a0a7d78'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.345407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79bea044ec2f451495ece1ff12b2b010', 'request_id': 'req-51036000-8ad0-440b-84de-e2353a0a7d78'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.348033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:26.356828 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a35d37-d122-47ce-a17a-b4c30ebdc82e None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:26.357510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] The token cache is being invalidate because project 79bea044ec2f451495ece1ff12b2b010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:26.357761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.358487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.360329 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1324/2639] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:26 2022] DELETE /identity/v3/projects/79bea044ec2f451495ece1ff12b2b010 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:26.363920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.365250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.407568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:26.408196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.408595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.408873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.410720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.429117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51036000-8ad0-440b-84de-e2353a0a7d78 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:26.433940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:26.434572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.435105 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.435391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] PATH_INFO: `/v3/users/fe80488e8f7f4b968eb526af83fb1cbb` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.436575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe80488e8f7f4b968eb526af83fb1cbb)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:26.438630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:26.476201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:26.477642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:26.486203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe80488e8f7f4b968eb526af83fb1cbb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.487645 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1316/2640] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:26.494148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe80488e8f7f4b968eb526af83fb1cbb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.508235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe80488e8f7f4b968eb526af83fb1cbb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.508917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe80488e8f7f4b968eb526af83fb1cbb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.509545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe80488e8f7f4b968eb526af83fb1cbb'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.511520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:26.525034 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1325/2641] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:26 2022] DELETE /identity/v3/users/fe80488e8f7f4b968eb526af83fb1cbb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:26.803866 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.806415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.822687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6346ef-a196-4773-adcf-785a42cf1f52 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:26.833122 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.834726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.894399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:26.895720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.896017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.896282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] PATH_INFO: `/v3/projects/14f76e0c775d4b1e8427cd4440a8bb1a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.897402 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] RBAC: Authorizing `identity:delete_project(project_id=14f76e0c775d4b1e8427cd4440a8bb1a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:26.899306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:26.903011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde1270-8c5e-4834-a2eb-4ff32d198504 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:26.908488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:26.909115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.909403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.909680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] PATH_INFO: `/v3/projects/087c11aa849b46f89414139ed46fb748` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:26.910831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=087c11aa849b46f89414139ed46fb748)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:26.912798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:26.945612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14f76e0c775d4b1e8427cd4440a8bb1a', 'request_id': 'req-da8355ac-a6f8-46b4-82e4-b0ac969e310b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.949470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:26.953362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14f76e0c775d4b1e8427cd4440a8bb1a', 'request_id': 'req-da8355ac-a6f8-46b4-82e4-b0ac969e310b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.960446 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14f76e0c775d4b1e8427cd4440a8bb1a', 'request_id': 'req-da8355ac-a6f8-46b4-82e4-b0ac969e310b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.961080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '087c11aa849b46f89414139ed46fb748', 'request_id': 'req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.967732 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '087c11aa849b46f89414139ed46fb748', 'request_id': 'req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.971709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] The token cache is being invalidate because project 14f76e0c775d4b1e8427cd4440a8bb1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:26.971997 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.972945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.974346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '087c11aa849b46f89414139ed46fb748', 'request_id': 'req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.974566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1317/2642] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:26 2022] DELETE /identity/v3/projects/14f76e0c775d4b1e8427cd4440a8bb1a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:26.978532 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:26.980394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:26.985190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] The token cache is being invalidate because project 087c11aa849b46f89414139ed46fb748 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:26.985481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.986123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dddb64be-ee94-4d5f-a6fd-bc4ea18902ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:26.987623 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1326/2643] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:26 2022] DELETE /identity/v3/projects/087c11aa849b46f89414139ed46fb748 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:26.993432 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec6be-5bf5-45f2-8ae0-2592c880c020 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:26.993751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec6be-5bf5-45f2-8ae0-2592c880c020 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:26.994030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aec6be-5bf5-45f2-8ae0-2592c880c020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.014161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-71aec6be-5bf5-45f2-8ae0-2592c880c020 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:27.015363 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-71aec6be-5bf5-45f2-8ae0-2592c880c020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.050641 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8355ac-a6f8-46b4-82e4-b0ac969e310b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.077345 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1327/2644] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:27.085530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.085822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.086099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.105070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:27.106275 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.109795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.110422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.110700 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.110974 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] PATH_INFO: `/v3/users/f314001cf4e24357bc0aecdf8c18dceb` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.112125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f314001cf4e24357bc0aecdf8c18dceb)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.114043 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.114407 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1328/2645] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:27.118457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.127342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8205dc06-e7a5-409b-88f6-7a4ae313113b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.132974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.133525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.133806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.134077 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.134715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:27.135019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.136652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.149718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1329/2646] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:07:27.154190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.159690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:27.163626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f314001cf4e24357bc0aecdf8c18dceb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.164695 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e9c72-01d8-4a04-88d5-99c3d4cfafd7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.170394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.171189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.172010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.172290 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.172942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.173748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f314001cf4e24357bc0aecdf8c18dceb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.174701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f314001cf4e24357bc0aecdf8c18dceb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.174889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.188699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f314001cf4e24357bc0aecdf8c18dceb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.189391 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f314001cf4e24357bc0aecdf8c18dceb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.202349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1318/2647] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:26 2022] DELETE /identity/v3/users/f314001cf4e24357bc0aecdf8c18dceb => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:27.204263 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1330/2648] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:27.208272 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.209848 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.220988 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:27.286006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a083ef-58e8-4a86-9783-a5ef34ee82bd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.342744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.343354 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.343675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.343957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.344589 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.346498 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.375368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1319/2649] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:27.376596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.387257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb7ab10-8822-4638-9cdd-73ba8288db43 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.392304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.392911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.393197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.393578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.394227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.396424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.407383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1331/2650] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:27.411930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.422421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b551a16-9c97-460c-a0b2-7ea17b8f0139 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.427422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.428144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.428341 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.428618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.429268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.430946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.441661 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1320/2651] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:27.446015 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.456736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78767b20-3b7f-469e-91f9-61e5fcec9607 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.462790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6853cad-9adc-4477-a89a-12e176d9c77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.463492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6853cad-9adc-4477-a89a-12e176d9c77a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.463809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6853cad-9adc-4477-a89a-12e176d9c77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.464152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6853cad-9adc-4477-a89a-12e176d9c77a admin admin] PATH_INFO: `/v3/projects/4bf05e3664334cce83f1aec3adf17003/users/90c12677653b42a9aa535e152773216d/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.480967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6853cad-9adc-4477-a89a-12e176d9c77a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bf05e3664334cce83f1aec3adf17003, user_id=90c12677653b42a9aa535e152773216d, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.485700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6853cad-9adc-4477-a89a-12e176d9c77a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.486719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.488823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.503621 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1332/2652] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:27 2022] PUT /identity/v3/projects/4bf05e3664334cce83f1aec3adf17003/users/90c12677653b42a9aa535e152773216d/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:27.513693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.513809 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8d3e8-8e74-499f-b869-968852e45bc3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.513977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.514191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.542724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] MFA Rules not processed for user `90c12677653b42a9aa535e152773216d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:27.543985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.573664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.574233 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.574517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.574799 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] PATH_INFO: `/v3/projects/d6a4602ec81f4cb3a0ef193138bc3990` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.575978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6a4602ec81f4cb3a0ef193138bc3990)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.577874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.609107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1333/2653] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:27.613334 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.620339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:27.624027 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a08b366-c9bf-4a6e-8356-ace2e0ed34d1 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.632613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6a4602ec81f4cb3a0ef193138bc3990', 'request_id': 'req-45357dc7-7a35-4770-85e8-8cf7a580b9df'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.640459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6a4602ec81f4cb3a0ef193138bc3990', 'request_id': 'req-45357dc7-7a35-4770-85e8-8cf7a580b9df'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.646862 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6a4602ec81f4cb3a0ef193138bc3990', 'request_id': 'req-45357dc7-7a35-4770-85e8-8cf7a580b9df'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.658786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] The token cache is being invalidate because project d6a4602ec81f4cb3a0ef193138bc3990 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:27.659305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.660083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-45357dc7-7a35-4770-85e8-8cf7a580b9df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:27.661670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1321/2654] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:27 2022] DELETE /identity/v3/projects/d6a4602ec81f4cb3a0ef193138bc3990 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:27.666922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad163b-e77d-43ce-b7cf-13b273ac9545 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.667813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad163b-e77d-43ce-b7cf-13b273ac9545 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.668291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad163b-e77d-43ce-b7cf-13b273ac9545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.686568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.687191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.687475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.687785 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.688425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.690340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.690546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-99ad163b-e77d-43ce-b7cf-13b273ac9545 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:27.692294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-99ad163b-e77d-43ce-b7cf-13b273ac9545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.700796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1322/2655] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:27.709420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.709703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.709976 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.717471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1334/2656] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:27.721120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.723605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.733634 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:27.734830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:27.745825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:27.791196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c34b08-0c64-4783-9a62-067523262067 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.795119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1323/2657] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:27.795961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.796575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.796859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.797141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.797804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.799309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.800055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.808851 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00782c3-bb1a-42be-83c6-c1b297839249 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.814607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.815320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.815679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.815936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.816682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:27.817031 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.818623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.827208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1335/2658] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:27.827208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.833990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1324/2659] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 07 16:07:27.834993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.837261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f21d7d-aef2-4186-8a09-02baf0931a4a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.842013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.842611 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.842890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.843165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.843867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.844569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e7205-fb63-41f5-bf76-af739bf75f1e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.845466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.850179 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.850769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.851054 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.851327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.852002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.853935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.857060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1336/2660] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:27.860927 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.875062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289fd48-6abf-4164-9cb6-cd539cecea0a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.880346 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.880957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.881243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.881525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.882168 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.883871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.888230 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1325/2661] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:27.889250 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.894803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1337/2662] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:27.899461 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.909166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9d318-4877-4d59-83a4-aa63820b1471 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.951169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6cbbf2-1fed-4287-b913-d52cd29d7a7f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:27.956081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.956738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.957020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.957834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.958459 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.960415 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.966344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e319404d-b0eb-4c10-ace0-c703342dc7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:27.966990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319404d-b0eb-4c10-ace0-c703342dc7a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:27.967268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319404d-b0eb-4c10-ace0-c703342dc7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:27.967536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e319404d-b0eb-4c10-ace0-c703342dc7a6 admin admin] PATH_INFO: `/v3/projects/f7618a0fd8fd4bb7b7318388fdd40e56/users/f5479584bfd84e8fb53da818dd453492/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:27.983141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319404d-b0eb-4c10-ace0-c703342dc7a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7618a0fd8fd4bb7b7318388fdd40e56, user_id=f5479584bfd84e8fb53da818dd453492, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:27.983476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1326/2663] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:27 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:27.987369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:27.989269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e319404d-b0eb-4c10-ace0-c703342dc7a6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:27.997747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e48088e-6ee6-4063-aa30-af653502b2c7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.002639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.003242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.003521 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.003826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.004476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.006148 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.007013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1338/2664] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:27 2022] PUT /identity/v3/projects/f7618a0fd8fd4bb7b7318388fdd40e56/users/f5479584bfd84e8fb53da818dd453492/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:28.020699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.020926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1327/2665] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:28.021072 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.021241 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.025181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.030258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:28.037701 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204315bd-16bc-4081-81ea-a6216522248c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.052513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] MFA Rules not processed for user `f5479584bfd84e8fb53da818dd453492`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:28.053763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:28.099281 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.099844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.100126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.100398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.101067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.102664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.113384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1328/2666] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:28.117793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.122339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:28.127778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941674cb-8372-4e50-b609-4c54b96c2e8c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.128669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1339/2667] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:28.132671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8b080-a556-4af1-89cb-17e2ae23c72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.133340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8b080-a556-4af1-89cb-17e2ae23c72a admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.133621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8b080-a556-4af1-89cb-17e2ae23c72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.133894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8b080-a556-4af1-89cb-17e2ae23c72a admin admin] PATH_INFO: `/v3/projects/8196ddc683bf47c79f57ef57203e00da/users/e06269de47cd4c47ae2dd109ca742f28/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.135648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.146445 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdb50e-34b3-4a8c-9e87-e79035727c00 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.149796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8b080-a556-4af1-89cb-17e2ae23c72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8196ddc683bf47c79f57ef57203e00da, user_id=e06269de47cd4c47ae2dd109ca742f28, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.152496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8b080-a556-4af1-89cb-17e2ae23c72a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.152727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.153668 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.153950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.154224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.154952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.156899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.172525 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1329/2668] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:28 2022] PUT /identity/v3/projects/8196ddc683bf47c79f57ef57203e00da/users/e06269de47cd4c47ae2dd109ca742f28/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:28.183763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.184048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.184355 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.195298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1340/2669] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:28.196500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.217326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] MFA Rules not processed for user `e06269de47cd4c47ae2dd109ca742f28`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:28.218575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:28.265330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645ea17-b1c5-44c6-8b23-a8575e52fe65 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.270208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.270820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.271098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.271375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.272038 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.273959 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.289201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1330/2670] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:28.293990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.302325 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1341/2671] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:28.303324 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.307650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ad9bb-e83c-4768-9682-4a9f5677cc01 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.313838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.314246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb4c97-5678-4e4d-bd9f-360754198446 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.314456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.314758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.315042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.315768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.317741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.319309 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.319964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.320248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.320561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.321258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.323124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.336632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1342/2672] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:28.341151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.349509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1331/2673] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:28.352524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955210a-c3fb-41bf-8d1c-416c1f35c6f0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.353335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.407091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:28.419135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.419831 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.420130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.420399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] PATH_INFO: `/v3/projects/5610401eafe242e48fe63967911e4588/users/b9f781058f7b43dda2497a2e19d651fd/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.428083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58014ed-3afa-4521-acbc-9630f44e311e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.435522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5610401eafe242e48fe63967911e4588, user_id=b9f781058f7b43dda2497a2e19d651fd, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.437121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.437725 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.438003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.438274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.438612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.438900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.440820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.457953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1343/2674] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:28 2022] PUT /identity/v3/projects/5610401eafe242e48fe63967911e4588/users/b9f781058f7b43dda2497a2e19d651fd/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:28.461763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.462457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1332/2675] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/users => generated 296 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:28.466191 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.481894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f32b142-3fcb-4875-b3ac-5fc8bad40faf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.511710 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:28.534376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3a1e8-40a0-4528-ba9c-b313c4c13ad5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.540817 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.541696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.542004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.542278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.543349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.545426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.545769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.546384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.546668 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.546938 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.547639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.549228 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.556443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1344/2676] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:28.560021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1333/2677] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:28.560648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.563873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.572674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8701df-33ed-48ac-8220-002cd1f91449 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.574824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ead29-ccc3-4be2-959d-85616d646cfc admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.577420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01358c9-b33b-4948-822b-e503ddc7f4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.578086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01358c9-b33b-4948-822b-e503ddc7f4de admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.578410 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01358c9-b33b-4948-822b-e503ddc7f4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.578638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01358c9-b33b-4948-822b-e503ddc7f4de admin admin] PATH_INFO: `/v3/projects/5610401eafe242e48fe63967911e4588/users/b9f781058f7b43dda2497a2e19d651fd/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.579566 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.580333 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.580606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01358c9-b33b-4948-822b-e503ddc7f4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=5610401eafe242e48fe63967911e4588, user_id=b9f781058f7b43dda2497a2e19d651fd, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.580712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.580872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] PATH_INFO: `/v3/projects/77b4146f7ff342a78f9a6f61b4bab8c3/users/d7b635e3fb6448b9b038f06e989fea1b/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.583248 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01358c9-b33b-4948-822b-e503ddc7f4de admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.595833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1345/2678] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:28 2022] PUT /identity/v3/projects/5610401eafe242e48fe63967911e4588/users/b9f781058f7b43dda2497a2e19d651fd/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:28.597712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b4146f7ff342a78f9a6f61b4bab8c3, user_id=d7b635e3fb6448b9b038f06e989fea1b, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.600972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.607425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.607814 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.608100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.615572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1334/2679] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:28 2022] PUT /identity/v3/projects/77b4146f7ff342a78f9a6f61b4bab8c3/users/d7b635e3fb6448b9b038f06e989fea1b/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:28.619401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.640530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] MFA Rules not processed for user `b9f781058f7b43dda2497a2e19d651fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:28.641736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:28.691664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42210281-f8b2-4296-9f11-d8b8174b6317 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.696489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.697112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.697394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.697666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.698318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.699956 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.714134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1335/2680] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:28.715115 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.725825 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d196495-20b0-4a7e-93ff-ad6a31fbba50 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.731461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189de9b-749f-43d6-8fdc-54fcd255d2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.732150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189de9b-749f-43d6-8fdc-54fcd255d2da admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.732280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1346/2681] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:28.732523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189de9b-749f-43d6-8fdc-54fcd255d2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.732775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189de9b-749f-43d6-8fdc-54fcd255d2da admin admin] PATH_INFO: `/v3/projects/77b4146f7ff342a78f9a6f61b4bab8c3/users/d7b635e3fb6448b9b038f06e989fea1b/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.734953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189de9b-749f-43d6-8fdc-54fcd255d2da admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b4146f7ff342a78f9a6f61b4bab8c3, user_id=d7b635e3fb6448b9b038f06e989fea1b, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.737717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189de9b-749f-43d6-8fdc-54fcd255d2da admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.745047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:28.746448 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:28.751140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1336/2682] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:28 2022] PUT /identity/v3/projects/77b4146f7ff342a78f9a6f61b4bab8c3/users/d7b635e3fb6448b9b038f06e989fea1b/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:28.761660 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.761941 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.762212 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.812907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6df99e-0c7f-4d57-87f4-336bfd38bf4e None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:28.867153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:28.867847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:28.868002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:28.868274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:28.870421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:28.891139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:28.917053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] MFA Rules not processed for user `d7b635e3fb6448b9b038f06e989fea1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:28.918277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:28.931397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:28.934829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:28.936344 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:28.946081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1347/2683] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:28.990658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1337/2684] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:29.001608 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:29.003098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:29.014165 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d160e-d3de-4aed-8fe5-8031702fe757 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:29.017807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d169c-ddb9-427b-abee-d70dde3d9498 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:29.018403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d169c-ddb9-427b-abee-d70dde3d9498 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:29.018681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d169c-ddb9-427b-abee-d70dde3d9498 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:29.018952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d169c-ddb9-427b-abee-d70dde3d9498 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:29.028336 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d169c-ddb9-427b-abee-d70dde3d9498 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:29.029832 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704d169c-ddb9-427b-abee-d70dde3d9498 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:29.039984 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1348/2685] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:29.441028 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:29.502146 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae66ce-d3ab-49d3-97f8-d26fe167d6d0 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:29.507438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:29.508099 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:29.508379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:29.508654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] PATH_INFO: `/v3/users/e06269de47cd4c47ae2dd109ca742f28` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:29.509772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e06269de47cd4c47ae2dd109ca742f28)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:29.511763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:29.555942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e06269de47cd4c47ae2dd109ca742f28'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:29.563107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e06269de47cd4c47ae2dd109ca742f28'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:29.563771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e06269de47cd4c47ae2dd109ca742f28'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:29.576009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e06269de47cd4c47ae2dd109ca742f28'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:29.576658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e06269de47cd4c47ae2dd109ca742f28'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:29.588795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1338/2686] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:29 2022] DELETE /identity/v3/users/e06269de47cd4c47ae2dd109ca742f28 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:29.593341 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:29.593633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:29.593914 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:29.612848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:29.614557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:29.672303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1349/2687] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:29.682441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:29.684308 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:29.719840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:29.745453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd858b-54e2-4056-bfc3-fb52c645c4d2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:29.749135 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:29.749730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:29.750005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:29.750268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:29.759482 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:29.761018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:29.770747 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1339/2688] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:30.289425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:30.291857 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:30.319245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad940857-0b05-4681-887f-1fcb8c1ddbd2 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:30.327277 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:30.328025 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:30.328372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:30.328712 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] PATH_INFO: `/v3/projects/8196ddc683bf47c79f57ef57203e00da` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:30.329891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8196ddc683bf47c79f57ef57203e00da)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:30.332398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:30.385083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8196ddc683bf47c79f57ef57203e00da', 'request_id': 'req-71dda6c5-98be-4e37-a73f-5e7682258cdf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.393085 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8196ddc683bf47c79f57ef57203e00da', 'request_id': 'req-71dda6c5-98be-4e37-a73f-5e7682258cdf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.399627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8196ddc683bf47c79f57ef57203e00da', 'request_id': 'req-71dda6c5-98be-4e37-a73f-5e7682258cdf'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.410774 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] The token cache is being invalidate because project 8196ddc683bf47c79f57ef57203e00da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:30.411066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.411890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.413970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1350/2689] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:30 2022] DELETE /identity/v3/projects/8196ddc683bf47c79f57ef57203e00da => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:30.420526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:30.430675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af759c6b-3751-4fa8-b20c-b23d393a528e service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:30.483266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:30.484685 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:30.489361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:30.489991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:30.490276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:30.490550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] PATH_INFO: `/v3/users/d7b635e3fb6448b9b038f06e989fea1b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:30.491707 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7b635e3fb6448b9b038f06e989fea1b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:30.493637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:30.518044 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:30.542635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7b635e3fb6448b9b038f06e989fea1b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.546418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dda6c5-98be-4e37-a73f-5e7682258cdf admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:30.550686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:30.551480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7b635e3fb6448b9b038f06e989fea1b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.552064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:30.552204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7b635e3fb6448b9b038f06e989fea1b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.552349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:30.552627 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:30.554496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:30.564325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7b635e3fb6448b9b038f06e989fea1b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.564971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7b635e3fb6448b9b038f06e989fea1b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.577276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1340/2690] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:30 2022] DELETE /identity/v3/users/d7b635e3fb6448b9b038f06e989fea1b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:07:30.616469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:30.618186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8250d5b-e201-44ea-ab60-593b0e4c0818 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:30.627909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1351/2691] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:30.852469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:30.854201 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:30.912740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:30.927017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ef22-c067-40ab-9271-8b4af19802b9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:30.932001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:30.932646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:30.932916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:30.933188 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] PATH_INFO: `/v3/projects/77b4146f7ff342a78f9a6f61b4bab8c3` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:30.934361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b4146f7ff342a78f9a6f61b4bab8c3)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:30.936296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:30.985819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b4146f7ff342a78f9a6f61b4bab8c3', 'request_id': 'req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.992689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b4146f7ff342a78f9a6f61b4bab8c3', 'request_id': 'req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:30.998443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b4146f7ff342a78f9a6f61b4bab8c3', 'request_id': 'req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:31.011109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] The token cache is being invalidate because project 77b4146f7ff342a78f9a6f61b4bab8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:31.011395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:31.012059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8b2cd853-85e3-4f1d-9f09-00a70d7fec6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:31.013480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1341/2692] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:30 2022] DELETE /identity/v3/projects/77b4146f7ff342a78f9a6f61b4bab8c3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:31.018546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.018840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.019172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.040742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:31.042146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:31.109428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1352/2693] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:31.118423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.118783 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.119132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.139003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:31.140296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:31.148808 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1342/2694] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:31.153335 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.164361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910a1f79-d160-4c37-adde-d7d86a368e27 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.169195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.169813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.170089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.170359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.171008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:31.171320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.173123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.186799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1353/2695] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:31.191082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.200251 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4159f298-e6be-4a62-b6a4-3f3c8d8e9089 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.205167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.205735 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.206008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.206275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.206923 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.209368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.211145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.212950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:31.237081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1343/2696] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:31 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:31.241669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.270970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b36c0-f5f7-4b16-91b8-8fa234b203af admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.296340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:31.303870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143f8b7-0e7f-416e-bc86-9cbd2511aa3d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.308704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.309309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.309590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.309865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.310493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.312499 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.321109 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.321805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.322108 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.322310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.324155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:31.333399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1344/2697] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:31 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:31.337394 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.346807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa01ab2-71d9-4b2d-9800-5d59a2c36783 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.352431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.353014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.353296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.353569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.354209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.355811 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.366948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1345/2698] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:31.370708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.380185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4372f7d8-7ca2-4e5e-b62f-3e893f894453 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.384713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.385289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.385574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.385850 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.386476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.388086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.389239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.390672 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f104d4-13c9-4cba-b296-cd5ead175e63 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.397906 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1346/2699] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:31.401064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1354/2700] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:07:31.402790 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.413396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ae8f65-260e-4b5e-bf66-52976e5fd5e6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.418051 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.418699 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.418984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.419258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] PATH_INFO: `/v3/projects/8fa0ae8ad8554b498b4c5d4bc41edf28/users/e61be654dcff46fea3fe27676d2b957b/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.434350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fa0ae8ad8554b498b4c5d4bc41edf28, user_id=e61be654dcff46fea3fe27676d2b957b, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.437261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.446682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:31.452669 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1347/2701] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:31 2022] PUT /identity/v3/projects/8fa0ae8ad8554b498b4c5d4bc41edf28/users/e61be654dcff46fea3fe27676d2b957b/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:31.463663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.463951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.464235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.494533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] MFA Rules not processed for user `e61be654dcff46fea3fe27676d2b957b`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:31.495823 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:31.564639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1355/2702] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:31.576291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:31.630796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dd07b2-d031-4c2b-b420-69a221c819a6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:31.634484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513be5a-081f-4a87-8953-43c541a53266 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:31.635077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513be5a-081f-4a87-8953-43c541a53266 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:31.635356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513be5a-081f-4a87-8953-43c541a53266 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:31.635666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513be5a-081f-4a87-8953-43c541a53266 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:31.643805 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513be5a-081f-4a87-8953-43c541a53266 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:31.645757 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513be5a-081f-4a87-8953-43c541a53266 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:31.654363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1348/2703] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:38.426871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:38.428359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:38.505433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c948116c-9113-4d1e-ae02-cbaa21352ddd None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:38.510139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:38.510901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:38.511183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:38.511561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] PATH_INFO: `/v3/users/90c12677653b42a9aa535e152773216d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:38.512769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90c12677653b42a9aa535e152773216d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:38.514689 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:38.521681 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:38.562124 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90c12677653b42a9aa535e152773216d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:38.569011 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90c12677653b42a9aa535e152773216d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:38.580733 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90c12677653b42a9aa535e152773216d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:38.581371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90c12677653b42a9aa535e152773216d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:38.582480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90c12677653b42a9aa535e152773216d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:38.594430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1356/2704] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:38 2022] DELETE /identity/v3/users/90c12677653b42a9aa535e152773216d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:38.599677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:38.600255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:38.600255 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:38.619082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:38.620305 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:38.677304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1349/2705] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:38.687097 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:38.688506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:38.745271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0817d54-53d4-42d7-a627-b0940735d8b5 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:38.748867 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:38.749462 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:38.749739 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:38.750008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:38.758789 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:38.760239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:38.769940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1357/2706] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:39.029092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.030497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:39.042346 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d276db-f2cf-4b43-8fa5-ffd187bfc8fc None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:39.047015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:39.047670 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.047951 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.048224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] PATH_INFO: `/v3/projects/4bf05e3664334cce83f1aec3adf17003` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.049366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bf05e3664334cce83f1aec3adf17003)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:39.051294 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:39.098396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bf05e3664334cce83f1aec3adf17003', 'request_id': 'req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.104868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bf05e3664334cce83f1aec3adf17003', 'request_id': 'req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.110404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bf05e3664334cce83f1aec3adf17003', 'request_id': 'req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.121178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] The token cache is being invalidate because project 4bf05e3664334cce83f1aec3adf17003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:39.121469 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.122092 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.123767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1350/2707] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:39 2022] DELETE /identity/v3/projects/4bf05e3664334cce83f1aec3adf17003 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:39.127637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.137520 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d7cc60-9d28-4cea-be3b-e18e60ad6c28 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:39.192850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:39.193480 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.193763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.194032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] PATH_INFO: `/v3/users/f5479584bfd84e8fb53da818dd453492` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.195127 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5479584bfd84e8fb53da818dd453492)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:39.197090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:39.228358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:39.243505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5479584bfd84e8fb53da818dd453492'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.250494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5479584bfd84e8fb53da818dd453492'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.262842 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5479584bfd84e8fb53da818dd453492'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.263494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5479584bfd84e8fb53da818dd453492'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.264136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5479584bfd84e8fb53da818dd453492'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.275971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1358/2708] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:39 2022] DELETE /identity/v3/users/f5479584bfd84e8fb53da818dd453492 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:39.543931 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.545356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:39.559297 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2c64a1-4ce7-4557-9b51-4258708ce4f7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:39.580814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:39.619174 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:39.619903 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.620091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.620357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] PATH_INFO: `/v3/projects/f7618a0fd8fd4bb7b7318388fdd40e56` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.621455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7618a0fd8fd4bb7b7318388fdd40e56)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:39.623325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:39.668140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7618a0fd8fd4bb7b7318388fdd40e56', 'request_id': 'req-5cd70911-6bbb-4e56-99b6-9d551e3086a6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.675501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7618a0fd8fd4bb7b7318388fdd40e56', 'request_id': 'req-5cd70911-6bbb-4e56-99b6-9d551e3086a6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.682236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7618a0fd8fd4bb7b7318388fdd40e56', 'request_id': 'req-5cd70911-6bbb-4e56-99b6-9d551e3086a6'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.693461 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] The token cache is being invalidate because project f7618a0fd8fd4bb7b7318388fdd40e56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:39.693754 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.694451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.696066 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1351/2709] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:39 2022] DELETE /identity/v3/projects/f7618a0fd8fd4bb7b7318388fdd40e56 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:39.700188 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:39.701637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:39.714571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f5a4b0-0f5a-4f14-98d4-7c18c1e798f7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:39.770082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:39.770707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:39.770991 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:39.771273 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] PATH_INFO: `/v3/users/b9f781058f7b43dda2497a2e19d651fd` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:39.772420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f781058f7b43dda2497a2e19d651fd)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:39.774339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:39.820257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f781058f7b43dda2497a2e19d651fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.828271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f781058f7b43dda2497a2e19d651fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.839923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f781058f7b43dda2497a2e19d651fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.840557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f781058f7b43dda2497a2e19d651fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.841151 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a06eda9b-e17d-481f-b45d-c32fa6d10e89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f781058f7b43dda2497a2e19d651fd'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:39.853790 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1359/2710] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:39 2022] DELETE /identity/v3/users/b9f781058f7b43dda2497a2e19d651fd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:40.117074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.118513 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:40.131505 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd70911-6bbb-4e56-99b6-9d551e3086a6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.187003 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.187656 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.187946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.188204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] PATH_INFO: `/v3/projects/5610401eafe242e48fe63967911e4588` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.189306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5610401eafe242e48fe63967911e4588)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.191758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.238116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5610401eafe242e48fe63967911e4588', 'request_id': 'req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:40.245299 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5610401eafe242e48fe63967911e4588', 'request_id': 'req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:40.246501 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:40.252323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5610401eafe242e48fe63967911e4588', 'request_id': 'req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:40.264065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] The token cache is being invalidate because project 5610401eafe242e48fe63967911e4588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:40.264351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:40.264981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-169f7a5c-4fd4-4f22-beb7-0a5fd57a47bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:40.266473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1352/2711] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:40 2022] DELETE /identity/v3/projects/5610401eafe242e48fe63967911e4588 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:40.273218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.273599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.273877 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.295631 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:40.296888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:40.320650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:40.368699 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1360/2712] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:40.378180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.378395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.379183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.400257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:40.401493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:40.409457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1353/2713] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:40.413878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.423957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0d57-430c-4cc3-96fd-d5229a3a1da9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.428545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.429146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.429427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.429700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.430339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:40.430645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.432286 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.444280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1361/2714] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:40.448452 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.458130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebba65a-d72c-4044-994e-f9a7863399e4 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.462789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.463375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.463694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.463964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.464596 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.466465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.492989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1354/2715] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:40 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:40.496987 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.507485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d1d2e-f702-4f41-a305-6b8e88b7dc2d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.564652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.565299 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.565582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.565889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.566568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.568536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.589560 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1362/2716] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:40 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:40.593533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.604553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97172c2a-528a-4cc5-9a2f-ff2ad10e4607 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.609262 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.609887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.610168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.610444 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.611090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.612732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.623177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1355/2717] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:40.626939 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.636996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced46c4-d8d9-4a20-beee-377c9561f88c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.641778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda249d7-711f-4edc-825d-d3c5c05f8db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.642385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda249d7-711f-4edc-825d-d3c5c05f8db2 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.642669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda249d7-711f-4edc-825d-d3c5c05f8db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.642946 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda249d7-711f-4edc-825d-d3c5c05f8db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.643649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda249d7-711f-4edc-825d-d3c5c05f8db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.645246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda249d7-711f-4edc-825d-d3c5c05f8db2 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.656055 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1363/2718] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:40.660454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.670247 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ba10ca-527a-423b-847d-2e875e7099eb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.675276 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.676553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.676835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.677110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] PATH_INFO: `/v3/projects/266789e160554ec0a0a334304a0ad39d/users/22aa615f2c7c4c6eaa694f551e38436a/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.691312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=266789e160554ec0a0a334304a0ad39d, user_id=22aa615f2c7c4c6eaa694f551e38436a, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.693972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.706347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1356/2719] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:40 2022] PUT /identity/v3/projects/266789e160554ec0a0a334304a0ad39d/users/22aa615f2c7c4c6eaa694f551e38436a/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:40.711806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c52d7-4c66-4dfc-b7e8-766543d5ec00 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.712257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c52d7-4c66-4dfc-b7e8-766543d5ec00 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.712387 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c52d7-4c66-4dfc-b7e8-766543d5ec00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.735254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1c4c52d7-4c66-4dfc-b7e8-766543d5ec00 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:40.736493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1c4c52d7-4c66-4dfc-b7e8-766543d5ec00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:40.794971 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1364/2720] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:40.804590 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:40.806301 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:40.863829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6fb0a-c36e-4bb3-8eeb-5f765a5aec37 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:40.867377 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:40.868000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:40.868283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:40.868551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:40.877236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:40.879217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:40.888552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1357/2721] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:44.549746 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.550283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.550283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.579813 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] MFA Rules not processed for user `22aa615f2c7c4c6eaa694f551e38436a`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:44.580995 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:44.603306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:44.652582 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1365/2722] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:44.653693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.664748 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efeffa-3809-4547-b717-466d9816642a service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.670830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.671667 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.672001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.672522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.673406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.675885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.705714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1358/2723] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:44 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:44.710523 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.725164 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74232574-a1a5-4a8b-9d48-fd721699c1b3 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.788980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.789605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.789887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.790159 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.790797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.792757 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.823830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1366/2724] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:44 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:07:44.827871 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.839474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb7b3c3-3501-49c3-b5bf-4bafda918f3e admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.844268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.844875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.845155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.845478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.846123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.848198 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.859273 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1359/2725] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:44.863638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.873915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c741047a-e77d-420f-8f69-edd2be5b3766 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:44.879200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:44.879891 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:44.880167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:44.880451 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] PATH_INFO: `/v3/projects/e821597a56454269a77d3c529a03a2cf/users/690176f79d9d4df0ae1a4078428b1375/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:44.897140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e821597a56454269a77d3c529a03a2cf, user_id=690176f79d9d4df0ae1a4078428b1375, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:44.900058 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:44.915769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1367/2726] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:44 2022] PUT /identity/v3/projects/e821597a56454269a77d3c529a03a2cf/users/690176f79d9d4df0ae1a4078428b1375/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:44.919388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:44.931239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd30494-cdd0-4e56-8347-3998528f664b admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.012488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:45.022159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d194a8-2049-4fe9-acc0-9ca80c8ceadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.022944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d194a8-2049-4fe9-acc0-9ca80c8ceadb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.023398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d194a8-2049-4fe9-acc0-9ca80c8ceadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.023813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d194a8-2049-4fe9-acc0-9ca80c8ceadb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.024538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d194a8-2049-4fe9-acc0-9ca80c8ceadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.026356 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d194a8-2049-4fe9-acc0-9ca80c8ceadb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.038421 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1360/2727] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:45.043251 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:45.055596 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cd0a7b-91d1-44db-a5a5-0664e95c33e7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:45.060422 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:45.061090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:45.061371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:45.061637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] PATH_INFO: `/v3/projects/e821597a56454269a77d3c529a03a2cf/users/690176f79d9d4df0ae1a4078428b1375/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:45.063747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e821597a56454269a77d3c529a03a2cf, user_id=690176f79d9d4df0ae1a4078428b1375, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:45.066682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:45.078180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1368/2728] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:45 2022] PUT /identity/v3/projects/e821597a56454269a77d3c529a03a2cf/users/690176f79d9d4df0ae1a4078428b1375/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:49.664122 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d851dd-bbe2-408d-91f4-84831a021de0 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.664703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d851dd-bbe2-408d-91f4-84831a021de0 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.664860 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d851dd-bbe2-408d-91f4-84831a021de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.695378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d5d851dd-bbe2-408d-91f4-84831a021de0 None None] MFA Rules not processed for user `690176f79d9d4df0ae1a4078428b1375`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:49.696606 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d5d851dd-bbe2-408d-91f4-84831a021de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:49.773472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1361/2729] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:49.789656 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:49.791720 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:49.875736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3951173-6672-493c-b1bd-84ad1c5274b3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:49.881862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:49.887958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:49.887958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:49.887958 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:49.901163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:49.955177 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:49.957417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:49.967779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1369/2730] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:50.960924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f23715-4d95-48e5-9543-2c6ef651e375 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:50.961442 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f23715-4d95-48e5-9543-2c6ef651e375 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:50.961558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f23715-4d95-48e5-9543-2c6ef651e375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:50.980857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-d3f23715-4d95-48e5-9543-2c6ef651e375 None None] MFA Rules not processed for user `22aa615f2c7c4c6eaa694f551e38436a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:50.982067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d3f23715-4d95-48e5-9543-2c6ef651e375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:50.989173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1362/2731] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:55.848815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:55.850175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:55.916146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44450-ade3-4638-8a75-96177cf4e7ab service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:55.920642 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:55.921270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:55.921550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:55.921820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] PATH_INFO: `/v3/users/e61be654dcff46fea3fe27676d2b957b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:55.922919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e61be654dcff46fea3fe27676d2b957b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:55.924856 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:55.969450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e61be654dcff46fea3fe27676d2b957b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:55.977884 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e61be654dcff46fea3fe27676d2b957b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:55.992606 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e61be654dcff46fea3fe27676d2b957b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:55.993330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e61be654dcff46fea3fe27676d2b957b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:55.993930 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e61be654dcff46fea3fe27676d2b957b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:56.006380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1370/2732] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:07:55 2022] DELETE /identity/v3/users/e61be654dcff46fea3fe27676d2b957b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:56.010858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.011375 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.011487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.030020 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:56.031219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:56.092027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1363/2733] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:56.102494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:56.103843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:56.140417 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:56.166506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8e1936-eb53-4da5-902b-1535c05c299e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:56.170195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671eabc8-a3bc-4a92-a934-29e5c1668b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:56.170807 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eabc8-a3bc-4a92-a934-29e5c1668b7f service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.171091 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eabc8-a3bc-4a92-a934-29e5c1668b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.171368 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671eabc8-a3bc-4a92-a934-29e5c1668b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.181270 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671eabc8-a3bc-4a92-a934-29e5c1668b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:56.182709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671eabc8-a3bc-4a92-a934-29e5c1668b7f service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:56.192676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1371/2734] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:07:56.465637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:56.467065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:56.482549 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b239ee-4d4d-486e-8140-30727d9d581a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:56.497815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:56.498503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.498787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.499062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] PATH_INFO: `/v3/projects/8fa0ae8ad8554b498b4c5d4bc41edf28` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.500464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fa0ae8ad8554b498b4c5d4bc41edf28)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:56.502425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:56.550575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fa0ae8ad8554b498b4c5d4bc41edf28', 'request_id': 'req-b1673c66-d794-4c06-a994-f5e99d675fba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:56.561637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fa0ae8ad8554b498b4c5d4bc41edf28', 'request_id': 'req-b1673c66-d794-4c06-a994-f5e99d675fba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:56.566090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:56.568447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fa0ae8ad8554b498b4c5d4bc41edf28', 'request_id': 'req-b1673c66-d794-4c06-a994-f5e99d675fba'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:56.581826 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] The token cache is being invalidate because project 8fa0ae8ad8554b498b4c5d4bc41edf28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:07:56.582107 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:56.582741 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-b1673c66-d794-4c06-a994-f5e99d675fba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:07:56.584240 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1364/2735] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:07:56 2022] DELETE /identity/v3/projects/8fa0ae8ad8554b498b4c5d4bc41edf28 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:56.590074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.590358 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.590639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.610418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:56.611593 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:56.672028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1372/2736] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:07:56.738639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.738929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.739199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.759776 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:56.760978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:56.771428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1365/2737] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:56.775166 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:56.786132 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c7f62-eb13-47bd-896d-6898b2df37ee None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:56.791014 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:56.791428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.791869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.792131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.792643 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:07:56.792952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:56.794599 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:56.807571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1373/2738] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:07:56.811837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:56.821108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddb881c-07c7-4f7a-8d7b-e6b72c58031f None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:56.825734 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:56.826326 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.826621 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.826875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.827642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:56.829546 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:56.859378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1366/2739] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:07:56 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:56.864316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:56.874843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7e35c-2163-4051-ae1e-9c4e8090c17c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:56.934353 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:56.935043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.935340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.935724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.936477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:56.938440 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:56.959404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1374/2740] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:07:56 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:07:56.963716 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:56.974471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420c7bf8-f6a8-49c1-ac1e-433e954b3df1 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:56.979076 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:56.979703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:56.979986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:56.980258 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:56.980899 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:56.982474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:56.992773 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1367/2741] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:07:56.996854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:57.008468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139edfd-cbf0-4040-a3d1-7572e5c6ce91 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:57.013332 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb061e48-0e87-41a8-86d5-7829e7a675a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:57.013949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb061e48-0e87-41a8-86d5-7829e7a675a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.014228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb061e48-0e87-41a8-86d5-7829e7a675a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.014501 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb061e48-0e87-41a8-86d5-7829e7a675a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.015152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb061e48-0e87-41a8-86d5-7829e7a675a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:57.016791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb061e48-0e87-41a8-86d5-7829e7a675a0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:57.027845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1375/2742] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:07:57.032141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:57.042686 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b5e4d-9922-42a7-a84a-68bf210fa753 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:57.047283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:57.047965 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.048253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.048523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] PATH_INFO: `/v3/projects/bf3370f177064dce9378221ca064731d/users/1d0a3350affa4659a7c7098806d43580/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.063351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf3370f177064dce9378221ca064731d, user_id=1d0a3350affa4659a7c7098806d43580, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:57.066581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:57.080710 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1368/2743] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:07:57 2022] PUT /identity/v3/projects/bf3370f177064dce9378221ca064731d/users/1d0a3350affa4659a7c7098806d43580/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:07:57.091249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.091558 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.091837 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.119092 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] MFA Rules not processed for user `1d0a3350affa4659a7c7098806d43580`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:07:57.120298 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:57.151392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:07:57.188493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1376/2744] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:07:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:07:57.200898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:07:57.202782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:07:57.283935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab2600-70a1-492b-a37e-a84d52d43a61 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:07:57.288229 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '257da2327d944095b51df9f677b3a99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:07:57.288857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:07:57.289141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:07:57.289413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:07:57.297455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:07:57.298874 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:07:57.307420 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1369/2745] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:07:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:07.185561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=14 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:07.303531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=1,keystone:SELECT=66,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:11.443266 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:11.444707 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:11.505095 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa5e4e-b4ec-40c0-87b0-07767163ada8 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:11.508931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:11.509548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:11.509838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:11.510120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:11.512086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:11.581333 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:11.582772 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:11.591561 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1377/2746] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:21.587516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=48 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:22.981689 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:22.983373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:23.096519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a6508-7dd2-4c70-8510-494b6aee7339 service glance] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:23.103681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:23.104564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:23.105177 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:23.119698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:23.119698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:23.140706 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:23.143347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa4c3e-f5ca-41f1-a589-922fb3354097 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:23.158780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1370/2747] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:26.643840 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:26.645239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:26.700964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18d09f-ccc5-410e-837d-d8cc733d4778 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:26.704547 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:26.705147 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:26.705429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:26.705700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:26.707383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:26.717342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:26.718778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:26.726924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1378/2748] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:28.844005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:28.844545 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:28.844746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:28.865887 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] MFA Rules not processed for user `d595f32b5f2d406b9d6ce62acec1456b`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:28.867081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:28.873896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1371/2749] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:08:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:08:29.296729 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:29.298064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:29.308601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f10ba-8166-45b9-8134-c88d90090966 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:29.312285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:29.312900 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:29.313174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:29.313456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:29.315161 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:29.325359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:29.326798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f558a-d30b-4d7d-bf30-fa96a0e5899e service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:29.335736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1379/2750] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:32.461393 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:32.478112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291e96ce-6975-4fac-a5dd-d19b0cf0656a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:32.489577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:32.491134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:32.491603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:32.492040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:32.494080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:32.513221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:32.515892 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:32.529932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1372/2751] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:36.810999 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:36.811528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:36.811669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:36.831204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] MFA Rules not processed for user `2dab44e593194fe788216b029eecd10c`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:36.832450 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:36.890378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1380/2752] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:08:42.526438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=43 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:46.882848 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=57 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:47.176441 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:47.179637 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:47.193760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a2af2-bb3c-4195-aba0-2c5d31cc30d7 service nova] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:47.199265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:47.199957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:47.200238 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:47.200510 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:47.202213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:47.214730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:47.216347 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d7d192-5463-4a7b-ae0c-6542378964bf service nova] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:47.227694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1373/2753] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:57.222618 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:58.508530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:58.591795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadd72d-7b89-4f6a-aa44-115d68aacc71 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:58.596756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:58.597404 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:58.597683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:58.597954 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] PATH_INFO: `/v3/users/1d0a3350affa4659a7c7098806d43580` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:58.599669 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d0a3350affa4659a7c7098806d43580)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:58.601597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:58.672178 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d0a3350affa4659a7c7098806d43580'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:58.682594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d0a3350affa4659a7c7098806d43580'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:58.698175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d0a3350affa4659a7c7098806d43580'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:58.699041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d0a3350affa4659a7c7098806d43580'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:58.699756 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d0a3350affa4659a7c7098806d43580'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:58.715580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1381/2754] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:08:58 2022] DELETE /identity/v3/users/1d0a3350affa4659a7c7098806d43580 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:08:58.719939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:58.720439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:58.722296 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:58.743533 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:58.744824 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:58.821629 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1374/2755] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:08:58.829443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:58.830850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:58.831089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:58.832457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:58.890560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec5140d-b33c-4e8f-bc0c-f7924ad07781 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:58.894440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:58.895072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:58.895349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:58.895460 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652d758-3092-49eb-9d0a-48a274094ab6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:58.895742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:58.899043 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:58.899691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:58.899974 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:58.900250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:58.902032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:58.905966 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:58.907529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:58.918815 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1375/2756] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:08:58.938541 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:58.978000 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:58.979633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075d1756-bf3a-4346-b86b-6905e6068137 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:58.991646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1382/2757] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:08:59.225789 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.227584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:59.242978 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646f273-375d-4a17-b32f-4ba3cd165fa6 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.250359 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.251006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.251813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.252080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] PATH_INFO: `/v3/projects/bf3370f177064dce9378221ca064731d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.253319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf3370f177064dce9378221ca064731d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.255269 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.303934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf3370f177064dce9378221ca064731d', 'request_id': 'req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:59.312752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf3370f177064dce9378221ca064731d', 'request_id': 'req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:59.318882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf3370f177064dce9378221ca064731d', 'request_id': 'req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:59.330550 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] The token cache is being invalidate because project bf3370f177064dce9378221ca064731d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:08:59.330836 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:59.331487 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ec678b6f-8e9c-414a-94f8-23c73cafc0e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:08:59.333497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1376/2758] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:08:59 2022] DELETE /identity/v3/projects/bf3370f177064dce9378221ca064731d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:08:59.338498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.338873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.339222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.359488 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:59.360830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:59.422200 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1383/2759] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:08:59.431593 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.431778 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.432013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.454718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:59.456083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:59.468528 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1377/2760] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:08:59.469815 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.482820 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ec88d-b65b-4ecb-9ab5-f3ea66b16822 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.488018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.488638 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.489555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.489830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.490483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:08:59.490787 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.492443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.509376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1384/2761] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:08:59.510959 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.521447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda36b2-0255-424a-b355-a0181a15f177 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.526390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.527050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.527298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.527595 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.528926 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.531662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.566008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:59.575584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1378/2762] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:08:59 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:08:59.579728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.591374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e42d8-e5ec-4242-a1c7-4332e3a45e2d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.661584 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.662203 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.662477 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.662747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.663371 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.665391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.690742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1385/2763] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:08:59 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:08:59.692404 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.702345 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995e5867-f875-489d-b53b-2aca002f23aa admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.706830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.708087 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.708551 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.708804 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.709445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.711014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.720943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1379/2764] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:08:59.725580 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.735530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e21f7c-5404-4dbc-a9bb-ecf00ba7f893 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.740730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36952fc-5fd5-41be-bc9a-8895ea46b95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.741436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36952fc-5fd5-41be-bc9a-8895ea46b95f admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.741718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36952fc-5fd5-41be-bc9a-8895ea46b95f admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.742020 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36952fc-5fd5-41be-bc9a-8895ea46b95f admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.742674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36952fc-5fd5-41be-bc9a-8895ea46b95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.744303 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36952fc-5fd5-41be-bc9a-8895ea46b95f admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.756049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1386/2765] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:08:59.760278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.773063 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f8a50e-cb4c-40c6-b44c-ad7e663f25de admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:08:59.778332 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:08:59.778992 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.779274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.779572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] PATH_INFO: `/v3/projects/515e28fe5dd64b73942e73d2fd6342c6/users/0823aec2ecdc42d4b60ebc6e430a5b24/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.798152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=515e28fe5dd64b73942e73d2fd6342c6, user_id=0823aec2ecdc42d4b60ebc6e430a5b24, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:08:59.800928 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:08:59.824617 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1380/2766] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:08:59 2022] PUT /identity/v3/projects/515e28fe5dd64b73942e73d2fd6342c6/users/0823aec2ecdc42d4b60ebc6e430a5b24/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:08:59.834797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:08:59.835156 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:08:59.835530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:08:59.864798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] MFA Rules not processed for user `0823aec2ecdc42d4b60ebc6e430a5b24`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:08:59.866686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:08:59.890980 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:08:59.933319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1387/2767] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:08:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:08:59.944555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:08:59.946190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:09:00.007147 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7025854-0f72-411a-a75d-1585616949f9 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:09:00.021460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:09:00.021460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:09:00.021460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:09:00.021460 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:09:00.026445 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:09:00.027909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:09:00.037169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1381/2768] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:09:04.042542 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:09:04.044100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:09:04.114554 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74cfa-73cb-4226-af7e-335312c1b831 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:09:04.118118 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:09:04.118841 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:09:04.119121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:09:04.119389 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:09:04.121262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:09:04.186847 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:09:04.188354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48347e51-4142-49af-84a9-f542f1abcb40 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:09:04.198767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1388/2769] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:09:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:09:10.032849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:09:10.302830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:09:10.366406 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131d6772-906a-4b38-a645-a3ca1b092b68 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:09:10.371200 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:09:10.371870 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:09:10.372155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:09:10.372431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] PATH_INFO: `/v3/users/0823aec2ecdc42d4b60ebc6e430a5b24` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:09:10.373572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=0823aec2ecdc42d4b60ebc6e430a5b24)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:09:10.375484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:09:10.427553 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0823aec2ecdc42d4b60ebc6e430a5b24'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:10.434902 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0823aec2ecdc42d4b60ebc6e430a5b24'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:10.435947 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0823aec2ecdc42d4b60ebc6e430a5b24'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:10.449231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0823aec2ecdc42d4b60ebc6e430a5b24'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:10.449930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0823aec2ecdc42d4b60ebc6e430a5b24'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:10.462443 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1382/2770] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:09:10 2022] DELETE /identity/v3/users/0823aec2ecdc42d4b60ebc6e430a5b24 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:09:10.467005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:09:10.467499 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:09:10.467630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:09:10.487005 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:09:10.488961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:09:10.551323 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1389/2771] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:09:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:09:10.561061 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:09:10.562520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:09:10.625483 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c7557-a891-4ab5-b0e7-9bb2ab0c78ce admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:09:10.629160 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:09:10.629758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:09:10.630036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:09:10.630363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:09:10.640642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:09:10.642435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:09:10.652110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1383/2772] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:09:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:09:10.915083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:09:10.916637 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:09:10.931398 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33078e-7f60-451f-8cc9-41ac9e802149 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:09:10.936080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:09:10.936719 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:09:10.937010 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:09:10.937282 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] PATH_INFO: `/v3/projects/515e28fe5dd64b73942e73d2fd6342c6` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:09:10.938492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=515e28fe5dd64b73942e73d2fd6342c6)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:09:10.940511 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:09:10.952171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=100 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:09:10.990319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '515e28fe5dd64b73942e73d2fd6342c6', 'request_id': 'req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:10.998002 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '515e28fe5dd64b73942e73d2fd6342c6', 'request_id': 'req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:11.004897 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '515e28fe5dd64b73942e73d2fd6342c6', 'request_id': 'req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:11.015427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] The token cache is being invalidate because project 515e28fe5dd64b73942e73d2fd6342c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:09:11.015750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:11.016376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:09:11.017851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1390/2773] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:09:10 2022] DELETE /identity/v3/projects/515e28fe5dd64b73942e73d2fd6342c6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:09:20.647896 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:09:21.012439 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:10:14.116802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:10:14.118245 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:14.186318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858856d0-0e86-4e91-8d32-cd5b8e9e3d9f service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:10:14.244197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:10:14.244795 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:10:14.245069 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:10:14.245337 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:10:14.247046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:14.257819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:10:14.259369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372554bd-7e50-4328-b268-e8789dd6d4e2 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:10:14.268943 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1384/2774] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:10:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:10:24.264820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=58 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:10:33.098021 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:10:33.099658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:33.172443 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b9d65-9b17-4ee9-bef8-7e31a7a3ee4c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:10:33.178512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:10:33.179120 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:10:33.179395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:10:33.179704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:10:33.181420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:33.239100 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:10:33.240722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:10:33.249630 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1391/2775] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 07 16:10:43.245498 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=56 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:10:51.720192 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:10:51.720999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:10:51.721127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:10:51.740882 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] MFA Rules not processed for user `2dab44e593194fe788216b029eecd10c`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:10:51.742104 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:51.748855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1385/2776] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:10:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:10:53.470004 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:10:53.471706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:53.532469 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda6ca18-7220-408f-9dfb-a2e2a0f87a43 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:10:53.538213 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:10:53.539583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:10:53.539583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:10:53.539583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:10:53.541400 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:53.556594 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:10:53.558183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:10:53.571795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1392/2777] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:10:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:10:55.871516 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:10:55.872930 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:55.885431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b47f4-2c46-4c56-ab5c-bd63caf07273 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:10:55.888872 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:10:55.889455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:10:55.889729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:10:55.889995 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:10:55.891639 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:10:55.905049 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:10:55.906451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:10:55.917488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1386/2778] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:10:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:11:03.564688 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=26 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:11:05.912342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=9 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:11:23.523759 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:23.532686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430f61bf-7744-4a25-82cc-0d65b1f26856 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:23.536327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:23.536920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:23.537197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:23.537467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:23.539137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:23.601062 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:23.602545 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:23.612931 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1393/2779] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:11:26.610254 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:26.611752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:26.627065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e9d5a-3974-4950-8b77-5a170e59b659 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:26.632261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:26.632869 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:26.633153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:26.633439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:26.635155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:26.648207 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:26.649840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:26.661260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1387/2780] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:11:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:11:29.778086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:29.779781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:29.796117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8e229-a55b-416b-94e3-062ac3abe661 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:29.801280 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:29.801923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:29.802204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:29.802478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] PATH_INFO: `/v3/users/d595f32b5f2d406b9d6ce62acec1456b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:29.803670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d595f32b5f2d406b9d6ce62acec1456b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:29.805548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:29.858821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd595f32b5f2d406b9d6ce62acec1456b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:29.866692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd595f32b5f2d406b9d6ce62acec1456b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:29.881046 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd595f32b5f2d406b9d6ce62acec1456b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:29.881778 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd595f32b5f2d406b9d6ce62acec1456b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:29.882431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd595f32b5f2d406b9d6ce62acec1456b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:29.901249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1394/2781] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:11:29 2022] DELETE /identity/v3/users/d595f32b5f2d406b9d6ce62acec1456b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:11:30.069971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:30.080530 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245f510-ce35-4152-b131-0951dfecbf74 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:30.140217 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:30.140849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:30.141126 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:30.141396 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] PATH_INFO: `/v3/projects/6ef2c5a9809b4bee95a171cf6dca4e2b` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:30.142561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ef2c5a9809b4bee95a171cf6dca4e2b)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:30.145041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:30.189411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ef2c5a9809b4bee95a171cf6dca4e2b', 'request_id': 'req-d41cf463-a599-4905-b9a7-efadc80cad36'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.196971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ef2c5a9809b4bee95a171cf6dca4e2b', 'request_id': 'req-d41cf463-a599-4905-b9a7-efadc80cad36'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.202666 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ef2c5a9809b4bee95a171cf6dca4e2b', 'request_id': 'req-d41cf463-a599-4905-b9a7-efadc80cad36'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.214074 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] The token cache is being invalidate because project 6ef2c5a9809b4bee95a171cf6dca4e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:11:30.214363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.214984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.216481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1388/2782] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:11:30 2022] DELETE /identity/v3/projects/6ef2c5a9809b4bee95a171cf6dca4e2b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:11:30.220674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:30.222173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:30.235131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef9b37c-fe85-490d-80d5-c2a2b5085451 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:30.290467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:30.291093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:30.291372 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:30.291690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] PATH_INFO: `/v3/users/6c641d4b768041269181ae8b299fb10d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:30.292810 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c641d4b768041269181ae8b299fb10d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:30.294718 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:30.299833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:11:30.340928 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c641d4b768041269181ae8b299fb10d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.348283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c641d4b768041269181ae8b299fb10d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.360198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c641d4b768041269181ae8b299fb10d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.360860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c641d4b768041269181ae8b299fb10d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.361468 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c641d4b768041269181ae8b299fb10d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.374030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1395/2783] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:11:30 2022] DELETE /identity/v3/users/6c641d4b768041269181ae8b299fb10d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:11:30.546711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:30.548290 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:30.563623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41cf463-a599-4905-b9a7-efadc80cad36 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:30.620364 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:30.620915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:30.621194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:30.621466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] PATH_INFO: `/v3/projects/3d5e01b220564c38aeb39c88ac7ea530` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:30.622581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d5e01b220564c38aeb39c88ac7ea530)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:30.624511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:30.646116 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:11:30.671414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d5e01b220564c38aeb39c88ac7ea530', 'request_id': 'req-c2b159d9-d676-4161-90b6-6ef7dc520b86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.679265 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d5e01b220564c38aeb39c88ac7ea530', 'request_id': 'req-c2b159d9-d676-4161-90b6-6ef7dc520b86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.684954 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d5e01b220564c38aeb39c88ac7ea530', 'request_id': 'req-c2b159d9-d676-4161-90b6-6ef7dc520b86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.696844 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] The token cache is being invalidate because project 3d5e01b220564c38aeb39c88ac7ea530 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:11:30.697132 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.697764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c2b159d9-d676-4161-90b6-6ef7dc520b86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:11:30.699266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1389/2784] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:11:30 2022] DELETE /identity/v3/projects/3d5e01b220564c38aeb39c88ac7ea530 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:11:40.370682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:11:40.693535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:11:49.028070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:49.029500 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:49.085380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26adff3f-fe5f-4b9a-899f-025ea3fe3612 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:49.131903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:49.132495 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:49.132771 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:49.133041 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:49.134735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:49.143747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:49.145163 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c99d21a-b0da-40dc-a3bd-ba0d883d7cba service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:49.153544 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1396/2785] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:11:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:11:54.258398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40870c-f37d-42e3-a436-e38042f635f8 None None] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.258999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40870c-f37d-42e3-a436-e38042f635f8 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.258999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a40870c-f37d-42e3-a436-e38042f635f8 None None] PATH_INFO: `/` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.260544 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1390/2786] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 16:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:11:54.266529 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c40d9b-214f-4692-8771-50af6a93a0d2 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.267021 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c40d9b-214f-4692-8771-50af6a93a0d2 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.267142 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c40d9b-214f-4692-8771-50af6a93a0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.307906 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-36c40d9b-214f-4692-8771-50af6a93a0d2 None None] MFA Rules not processed for user `690506f288f84debaadb4c8975759dad`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:11:54.309125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-36c40d9b-214f-4692-8771-50af6a93a0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:54.316766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1397/2787] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:11:54.329855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40870c-f37d-42e3-a436-e38042f635f8 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:54.331715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-5a40870c-f37d-42e3-a436-e38042f635f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:54.344794 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a40870c-f37d-42e3-a436-e38042f635f8 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:54.348662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaefe72-8995-4229-9f2d-747b413ab8b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:54.349366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaefe72-8995-4229-9f2d-747b413ab8b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.349708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaefe72-8995-4229-9f2d-747b413ab8b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.350050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaefe72-8995-4229-9f2d-747b413ab8b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.357157 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaefe72-8995-4229-9f2d-747b413ab8b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:54.358713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaefe72-8995-4229-9f2d-747b413ab8b2 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:54.367484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1391/2788] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:11:54.456951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a25db-c61b-420a-ac62-c45c1e483b14 None None] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.457239 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a25db-c61b-420a-ac62-c45c1e483b14 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.457515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a25db-c61b-420a-ac62-c45c1e483b14 None None] PATH_INFO: `/` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.458933 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1398/2789] 10.222.0.17 () {54 vars in 921 bytes} [Wed Sep 7 16:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 07 16:11:54.464216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.464508 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.464791 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.501955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] MFA Rules not processed for user `8a9090ab0ffa4b7eafcc742880237229`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:11:54.503203 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:54.569193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1392/2790] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Sep 7 16:11:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:11:54.579489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a25db-c61b-420a-ac62-c45c1e483b14 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:54.590471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a25db-c61b-420a-ac62-c45c1e483b14 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:54.594023 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:54.594610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.594889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.595158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.605221 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:54.606633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:54.617673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1399/2791] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:11:54.678679 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:11:54.680138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:54.747041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e68ace-bc1a-4e3a-b31a-39f0c56e1d6a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:11:54.752053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:11:54.752644 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:11:54.752919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:11:54.753287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:11:54.754885 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:11:54.765936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:11:54.767486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:11:54.777218 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1393/2792] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:11:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:12:04.612766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=61 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:12:04.773125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=75 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:12:39.111064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:39.112535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:39.123607 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d8a20c-fff5-49ed-9dcc-7ae7b7aa1630 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:39.127102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:39.127731 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:39.128130 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:39.128408 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:39.130028 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:39.193082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:39.195064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:39.204531 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1400/2793] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:12:42.235905 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:42.237932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:42.251812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31d2bf-ae6a-424a-8261-54322b597331 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:42.257361 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:42.258304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.258897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.259388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.261718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:42.278009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:42.279714 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:42.289510 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:42.290955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:42.299257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1394/2794] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:12:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:12:42.304981 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d0c612-2d9e-45f0-a4b0-48da6e67de2d service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:42.311536 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:42.312202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.312464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.312797 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] PATH_INFO: `/v3/users/2dab44e593194fe788216b029eecd10c` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.313854 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dab44e593194fe788216b029eecd10c)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:42.315834 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:42.364033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dab44e593194fe788216b029eecd10c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.371244 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dab44e593194fe788216b029eecd10c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.382882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dab44e593194fe788216b029eecd10c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.383512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dab44e593194fe788216b029eecd10c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.384141 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a6e05ec4-83e7-478e-8d76-938aa7b7e52b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dab44e593194fe788216b029eecd10c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.395985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1401/2795] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:12:42 2022] DELETE /identity/v3/users/2dab44e593194fe788216b029eecd10c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:12:42.567467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:42.578261 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ea8659-7aa3-4507-93ed-dab5cc4306b1 service placement] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:42.641303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:42.641932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.642206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.642472 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] PATH_INFO: `/v3/projects/ab1963a1a6684af58f5c240787788b60` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.643638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1963a1a6684af58f5c240787788b60)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:42.645529 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:42.689366 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1963a1a6684af58f5c240787788b60', 'request_id': 'req-8ef79d89-9f8f-4e55-aea7-0a12e0743847'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.697221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1963a1a6684af58f5c240787788b60', 'request_id': 'req-8ef79d89-9f8f-4e55-aea7-0a12e0743847'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.702702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1963a1a6684af58f5c240787788b60', 'request_id': 'req-8ef79d89-9f8f-4e55-aea7-0a12e0743847'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.715008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] The token cache is being invalidate because project ab1963a1a6684af58f5c240787788b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:12:42.715295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.715944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-8ef79d89-9f8f-4e55-aea7-0a12e0743847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:12:42.717388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1395/2796] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:12:42 2022] DELETE /identity/v3/projects/ab1963a1a6684af58f5c240787788b60 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:12:42.722517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.722799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.723070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.742255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:12:42.743378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:42.801283 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1402/2797] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:12:42.809752 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.810037 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.810307 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.830210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:12:42.831414 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:42.839780 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1396/2798] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:12:42.843806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:42.853626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f15f9e-9959-4a00-b8dc-5d00d88bfc43 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:42.858210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:42.858799 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.859083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.859361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.860045 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:12:42.860361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:42.861963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:42.863430 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:12:42.875401 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1403/2799] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:12:42.879574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:42.889114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3a1980-2163-436f-9a74-bbfef178444b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:42.893684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:42.894266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.894568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.894838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.895468 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:42.897994 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:42.924040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1397/2800] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:12:42 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:12:42.928195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:42.937952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d188cae-d7a3-4398-8ea6-47be5aed62f0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:42.992937 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:42.993478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:42.993755 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:42.994026 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:42.994650 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:42.996589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.018143 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1404/2801] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:12:42 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:12:43.022431 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.032711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c79d4-18ac-4828-a1ec-e833cba6184f admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.037304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.037898 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.038180 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.038455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.039095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.040702 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.050464 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1398/2802] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:12:43.054816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.064612 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122eafb5-dc2b-487b-ac53-74b3cbd5541b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.069821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f90a97-9089-4d03-b76c-9e9ceb77a25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.070406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f90a97-9089-4d03-b76c-9e9ceb77a25a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.070683 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f90a97-9089-4d03-b76c-9e9ceb77a25a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.070955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f90a97-9089-4d03-b76c-9e9ceb77a25a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.071614 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f90a97-9089-4d03-b76c-9e9ceb77a25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.073184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f90a97-9089-4d03-b76c-9e9ceb77a25a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.083515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1405/2803] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:12:43.087642 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.098638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e90aa-fa9c-4760-b3f8-8207f12eeb16 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.103588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.104239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.104517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.104786 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] PATH_INFO: `/v3/projects/cc8088dad0c542cbb1cc722327216458/users/326cd0aaa2df46f2a0b144322a2a584e/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.120216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8088dad0c542cbb1cc722327216458, user_id=326cd0aaa2df46f2a0b144322a2a584e, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.122838 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.135900 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1399/2804] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:12:43 2022] PUT /identity/v3/projects/cc8088dad0c542cbb1cc722327216458/users/326cd0aaa2df46f2a0b144322a2a584e/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:12:43.146158 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34672863-755a-4862-b028-5c123b55fd99 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.146453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34672863-755a-4862-b028-5c123b55fd99 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.146734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34672863-755a-4862-b028-5c123b55fd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.174465 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-34672863-755a-4862-b028-5c123b55fd99 None None] MFA Rules not processed for user `326cd0aaa2df46f2a0b144322a2a584e`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:12:43.175892 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-34672863-755a-4862-b028-5c123b55fd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:43.239310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1406/2805] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:12:43.244321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.258123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:12:43.308467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d94553-b183-46eb-bfee-f656fc0b8dee admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.313096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.313696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.313977 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.314249 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.314873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.316835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.343879 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1400/2806] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:12:43 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:12:43.348271 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34672863-755a-4862-b028-5c123b55fd99 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.359522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34672863-755a-4862-b028-5c123b55fd99 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.414784 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.415535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.415910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.416187 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.416825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.418783 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.427714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:12:43.443808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1407/2807] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:12:43 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:12:43.445279 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.457319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f96676-981f-40c4-a4e9-adaf392734bf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.461942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.462543 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.462830 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.463090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.463765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.465341 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.476722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1401/2808] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:12:43.480957 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.492102 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3474e26a-a65e-40c8-b055-e4971a3c2e16 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.498538 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.499137 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.499424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.499742 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] PATH_INFO: `/v3/projects/891981876d5c4ab098fb0ceab28a5aee/users/751813b04fa3486c9c145673d0d7cd64/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.514613 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=891981876d5c4ab098fb0ceab28a5aee, user_id=751813b04fa3486c9c145673d0d7cd64, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.517316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.530037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1408/2809] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:12:43 2022] PUT /identity/v3/projects/891981876d5c4ab098fb0ceab28a5aee/users/751813b04fa3486c9c145673d0d7cd64/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:12:43.534119 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.544719 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b29e5ef-c9f8-4abf-8a67-ab3f8d253d35 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.600920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4efe18-6de1-4d8b-ab99-1790cf090ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.601519 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4efe18-6de1-4d8b-ab99-1790cf090ece admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.601810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4efe18-6de1-4d8b-ab99-1790cf090ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.602172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4efe18-6de1-4d8b-ab99-1790cf090ece admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.602803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4efe18-6de1-4d8b-ab99-1790cf090ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.604424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4efe18-6de1-4d8b-ab99-1790cf090ece admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.614041 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1402/2810] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:12:43 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:12:43.618878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.629319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128101d-5df2-44ed-b629-cd31791607ec admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.633967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.634622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.634904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.635181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] PATH_INFO: `/v3/projects/891981876d5c4ab098fb0ceab28a5aee/users/751813b04fa3486c9c145673d0d7cd64/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.637153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891981876d5c4ab098fb0ceab28a5aee, user_id=751813b04fa3486c9c145673d0d7cd64, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.639836 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.650094 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1409/2811] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:12:43 2022] PUT /identity/v3/projects/891981876d5c4ab098fb0ceab28a5aee/users/751813b04fa3486c9c145673d0d7cd64/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:12:43.660548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.660964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.661384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.693984 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] MFA Rules not processed for user `751813b04fa3486c9c145673d0d7cd64`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:12:43.695317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:43.763449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1403/2812] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:12:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:12:43.774639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:43.776434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:12:43.833702 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625a3f9-764e-48e9-9490-b968a4bbaa78 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:43.837869 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:43.838466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:43.838745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:43.839017 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:43.893018 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:43.894475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:43.903705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1410/2813] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:12:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:12:50.574542 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:12:50.584424 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f091f667-cda3-41a6-8890-5d793af599bf None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:12:50.588039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34e21d-5a83-4878-a9b9-d95a6fe0090c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:12:50.588635 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34e21d-5a83-4878-a9b9-d95a6fe0090c service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:12:50.588912 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34e21d-5a83-4878-a9b9-d95a6fe0090c service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:12:50.589182 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34e21d-5a83-4878-a9b9-d95a6fe0090c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:12:50.596184 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:12:50.598471 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34e21d-5a83-4878-a9b9-d95a6fe0090c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:12:50.600388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b34e21d-5a83-4878-a9b9-d95a6fe0090c service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:12:50.610300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1404/2814] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:12:50 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:12:53.899851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:00.606224 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:18.513808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:18.573943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614891b4-f82f-40b3-846e-8ca813389ed7 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:18.578362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:18.578977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:18.579255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:18.579526 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] PATH_INFO: `/v3/users/326cd0aaa2df46f2a0b144322a2a584e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:18.580651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=326cd0aaa2df46f2a0b144322a2a584e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:18.582550 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:18.626945 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326cd0aaa2df46f2a0b144322a2a584e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:18.634373 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326cd0aaa2df46f2a0b144322a2a584e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:18.645955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326cd0aaa2df46f2a0b144322a2a584e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:18.646583 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326cd0aaa2df46f2a0b144322a2a584e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:18.647210 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326cd0aaa2df46f2a0b144322a2a584e'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:18.659196 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1411/2815] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:18 2022] DELETE /identity/v3/users/326cd0aaa2df46f2a0b144322a2a584e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:18.664456 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:18.664968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:18.664968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:18.683834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:18.685016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:18.745568 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1405/2816] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:18.755231 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:18.757180 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:18.813294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f943d4c-df28-4a5e-b983-be67d9eed6bc admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:18.816919 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:18.817555 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:18.817835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:18.818139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:18.827453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:18.828915 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:18.838786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1412/2817] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:19.095371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:19.096810 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:19.108662 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73635122-336c-4c68-93ce-1ef84a541d6b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:19.113517 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:19.114183 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.114465 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.114743 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] PATH_INFO: `/v3/projects/cc8088dad0c542cbb1cc722327216458` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.116503 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8088dad0c542cbb1cc722327216458)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:19.119231 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:19.165371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8088dad0c542cbb1cc722327216458', 'request_id': 'req-27142fd4-9011-48fb-85d9-a61fc79b2edd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.173712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8088dad0c542cbb1cc722327216458', 'request_id': 'req-27142fd4-9011-48fb-85d9-a61fc79b2edd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.180127 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8088dad0c542cbb1cc722327216458', 'request_id': 'req-27142fd4-9011-48fb-85d9-a61fc79b2edd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.191046 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] The token cache is being invalidate because project cc8088dad0c542cbb1cc722327216458 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:19.191339 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.192017 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.193534 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1406/2818] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:19 2022] DELETE /identity/v3/projects/cc8088dad0c542cbb1cc722327216458 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:19.198019 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:19.207777 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a503a1cb-dbe4-4816-98e9-f17b081347af service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:19.225801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:19.265724 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:19.266351 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.266635 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.266909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] PATH_INFO: `/v3/users/751813b04fa3486c9c145673d0d7cd64` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.268052 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=751813b04fa3486c9c145673d0d7cd64)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:19.269963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:19.314376 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '751813b04fa3486c9c145673d0d7cd64'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.321411 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '751813b04fa3486c9c145673d0d7cd64'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.333384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '751813b04fa3486c9c145673d0d7cd64'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.334016 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '751813b04fa3486c9c145673d0d7cd64'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.334622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-61048665-77c6-4faa-a8b2-db4ed32c01b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '751813b04fa3486c9c145673d0d7cd64'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.347040 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1413/2819] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:19 2022] DELETE /identity/v3/users/751813b04fa3486c9c145673d0d7cd64 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:19.616507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:19.617972 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:19.632663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27142fd4-9011-48fb-85d9-a61fc79b2edd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:19.697763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:19.698558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.699018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.699390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] PATH_INFO: `/v3/projects/891981876d5c4ab098fb0ceab28a5aee` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.700840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891981876d5c4ab098fb0ceab28a5aee)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:19.703039 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:19.733497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:19.751474 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891981876d5c4ab098fb0ceab28a5aee', 'request_id': 'req-d23f4a87-5224-4beb-820e-240da7f221b1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.758704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891981876d5c4ab098fb0ceab28a5aee', 'request_id': 'req-d23f4a87-5224-4beb-820e-240da7f221b1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.764314 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891981876d5c4ab098fb0ceab28a5aee', 'request_id': 'req-d23f4a87-5224-4beb-820e-240da7f221b1'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.775649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] The token cache is being invalidate because project 891981876d5c4ab098fb0ceab28a5aee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:19.775934 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.776554 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d23f4a87-5224-4beb-820e-240da7f221b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:19.778009 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1407/2820] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:19 2022] DELETE /identity/v3/projects/891981876d5c4ab098fb0ceab28a5aee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:19.782818 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.783320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.783436 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.803320 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:19.804568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:19.863634 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1414/2821] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:19.871125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.871410 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.871728 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.891059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:19.892283 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:19.900652 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1408/2822] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:19.904658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:19.914803 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c66075-a9a7-46be-aff2-8d4ebff1e441 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:19.919464 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:19.920154 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.920434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.920703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.921338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:19.921640 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:19.923219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:19.934427 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1415/2823] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:19.938569 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:19.947449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d94c54-5a01-4a44-9c49-594c70e6c936 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:19.952237 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:19.952857 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:19.953140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:19.953409 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:19.954072 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:19.955993 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:19.983248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1409/2824] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:19 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:13:19.987195 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:19.996924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba3dc0-bf62-4185-95c8-f5d369dd2786 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.022938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:20.055338 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.055990 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.056276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.056557 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.057197 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.059374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.079920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1416/2825] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:19 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:20.084059 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.094914 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19825b46-9b9a-4a9d-84a0-ce3028c333ce admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.099492 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.100120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.100403 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.100676 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.101313 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.102890 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.112582 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1410/2826] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:20.116383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.126276 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13f2057-39e2-4a25-ad51-42ae3055ca44 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.131626 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638affc4-0603-41de-973e-e1db8242e913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.132272 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638affc4-0603-41de-973e-e1db8242e913 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.132553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638affc4-0603-41de-973e-e1db8242e913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.132883 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638affc4-0603-41de-973e-e1db8242e913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.133524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638affc4-0603-41de-973e-e1db8242e913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.135101 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638affc4-0603-41de-973e-e1db8242e913 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.146121 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1417/2827] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:20.150894 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.160821 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746c4a0-8efd-43a4-94f0-1eae4ed126a3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.165373 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.166016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.166291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.166558 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] PATH_INFO: `/v3/projects/2cd00d49f8c3496b94fd8be394bf3a17/users/e9facb72544f4e0aad5a0fd971ffb0fa/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.180922 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cd00d49f8c3496b94fd8be394bf3a17, user_id=e9facb72544f4e0aad5a0fd971ffb0fa, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.183535 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.197772 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1411/2828] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:20 2022] PUT /identity/v3/projects/2cd00d49f8c3496b94fd8be394bf3a17/users/e9facb72544f4e0aad5a0fd971ffb0fa/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:20.208169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.208457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.208737 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.236294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] MFA Rules not processed for user `e9facb72544f4e0aad5a0fd971ffb0fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:20.237487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:20.300705 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1418/2829] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:20.305137 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.365277 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd33c41-540b-4a10-9e84-fc85ab26519d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.369948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.370484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.370769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.371042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.371768 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.373651 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.399744 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1412/2830] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:20 2022] POST /identity/v3/projects => generated 367 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:20.403751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.414146 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b9dcb2-02d1-4500-b351-e46dfbdf05cc None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.468504 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.469111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.469388 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.469917 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.470530 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.472475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.493093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1419/2831] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:20 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:20.498156 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.509083 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c5fec-c777-4a89-95d0-e0c69b2ced8c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.513732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.514327 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.514613 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.514888 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.515594 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.517181 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.527520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1413/2832] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:20.531391 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.541131 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07baa587-83dc-4575-ae81-10e1a73d77d1 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.545693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.546330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.546603 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.546871 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] PATH_INFO: `/v3/projects/b3ddce603230418687d10627080ae784/users/7e2294ae694d4a16aec41f28d6515e06/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.562162 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ddce603230418687d10627080ae784, user_id=7e2294ae694d4a16aec41f28d6515e06, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.564821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.577779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1420/2833] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:20 2022] PUT /identity/v3/projects/b3ddce603230418687d10627080ae784/users/7e2294ae694d4a16aec41f28d6515e06/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:20.581765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.591477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d23544-9456-452e-a731-1adf8fca82e3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.610939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:20.650008 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77a6e1-31ad-4c6e-9181-d5ed26b2a06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.650520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a6e1-31ad-4c6e-9181-d5ed26b2a06b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.650797 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a6e1-31ad-4c6e-9181-d5ed26b2a06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.651062 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77a6e1-31ad-4c6e-9181-d5ed26b2a06b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.651726 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77a6e1-31ad-4c6e-9181-d5ed26b2a06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.653286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77a6e1-31ad-4c6e-9181-d5ed26b2a06b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.662709 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1414/2834] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:20.666851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.672367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:20.678963 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8f4b79-6c11-47f5-afa9-1a128e6789df admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.684001 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.684676 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.684955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.685228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] PATH_INFO: `/v3/projects/b3ddce603230418687d10627080ae784/users/7e2294ae694d4a16aec41f28d6515e06/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.687148 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3ddce603230418687d10627080ae784, user_id=7e2294ae694d4a16aec41f28d6515e06, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.689861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.706855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1421/2835] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:20 2022] PUT /identity/v3/projects/b3ddce603230418687d10627080ae784/users/7e2294ae694d4a16aec41f28d6515e06/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:20.769742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.770351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.770351 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.797939 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] MFA Rules not processed for user `7e2294ae694d4a16aec41f28d6515e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:20.799155 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:20.865221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1415/2836] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:20.877173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:20.879029 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:20.935224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf96716-679d-484e-aab5-026ca43a5dfb admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:20.938786 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:20.939375 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:20.939682 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:20.939956 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:20.948657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:20.950064 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:20.959193 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1422/2837] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:21.003721 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:21.005114 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:21.016949 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f78f7d-0309-4243-ac47-a509a6d24f41 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:21.020561 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22c489b-68ed-48bf-b62f-d7d2ad661b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:21.021152 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c489b-68ed-48bf-b62f-d7d2ad661b5c service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:21.021430 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c489b-68ed-48bf-b62f-d7d2ad661b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:21.021732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22c489b-68ed-48bf-b62f-d7d2ad661b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:21.080168 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c489b-68ed-48bf-b62f-d7d2ad661b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:21.081591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22c489b-68ed-48bf-b62f-d7d2ad661b5c service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:21.090214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1416/2838] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:24.365961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:24.431174 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02213bee-946f-4517-ab6d-d94951e6be67 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:24.435675 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:24.436304 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:24.436585 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:24.436860 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] PATH_INFO: `/v3/users/e9facb72544f4e0aad5a0fd971ffb0fa` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:24.437979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9facb72544f4e0aad5a0fd971ffb0fa)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:24.440049 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:24.486535 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9facb72544f4e0aad5a0fd971ffb0fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:24.494125 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9facb72544f4e0aad5a0fd971ffb0fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:24.506655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9facb72544f4e0aad5a0fd971ffb0fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:24.507296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9facb72544f4e0aad5a0fd971ffb0fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:24.507905 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9facb72544f4e0aad5a0fd971ffb0fa'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:24.519692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1423/2839] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:24 2022] DELETE /identity/v3/users/e9facb72544f4e0aad5a0fd971ffb0fa => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:24.524166 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189e664-00cf-4397-b228-60555610371d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:24.524681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189e664-00cf-4397-b228-60555610371d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:24.524796 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189e664-00cf-4397-b228-60555610371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:24.543458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-b189e664-00cf-4397-b228-60555610371d None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:24.544688 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b189e664-00cf-4397-b228-60555610371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:24.584349 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:24.603746 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1417/2840] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:24.613311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:24.614691 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:24.628497 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:24.680899 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b651651-5455-4dd3-9d84-318c43bfc8bd admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:24.684666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:24.685233 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:24.685515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:24.685791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:24.694828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:24.696319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:24.706649 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1424/2841] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:24.966723 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189e664-00cf-4397-b228-60555610371d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:24.968330 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b189e664-00cf-4397-b228-60555610371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:24.981242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189e664-00cf-4397-b228-60555610371d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:24.986130 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:24.986767 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:24.987045 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:24.987318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] PATH_INFO: `/v3/projects/2cd00d49f8c3496b94fd8be394bf3a17` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:24.988477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cd00d49f8c3496b94fd8be394bf3a17)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:24.990384 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.039526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cd00d49f8c3496b94fd8be394bf3a17', 'request_id': 'req-13c48832-654c-4c03-8e34-8658244c351a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.047454 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cd00d49f8c3496b94fd8be394bf3a17', 'request_id': 'req-13c48832-654c-4c03-8e34-8658244c351a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.054065 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cd00d49f8c3496b94fd8be394bf3a17', 'request_id': 'req-13c48832-654c-4c03-8e34-8658244c351a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.065484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] The token cache is being invalidate because project 2cd00d49f8c3496b94fd8be394bf3a17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:25.065774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.066425 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.067929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1418/2842] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:24 2022] DELETE /identity/v3/projects/2cd00d49f8c3496b94fd8be394bf3a17 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:25.071921 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:25.081342 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4c8ef9-aef8-474d-8ce1-0e6f0ac03106 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:25.138647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:25.139285 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.139617 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.139887 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] PATH_INFO: `/v3/users/7e2294ae694d4a16aec41f28d6515e06` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.141238 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2294ae694d4a16aec41f28d6515e06)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:25.143189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.190081 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2294ae694d4a16aec41f28d6515e06'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.196967 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2294ae694d4a16aec41f28d6515e06'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.209522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2294ae694d4a16aec41f28d6515e06'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.210171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2294ae694d4a16aec41f28d6515e06'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.210775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-7411798b-2ccf-4aca-a3c6-bcf021914bb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2294ae694d4a16aec41f28d6515e06'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.222970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1425/2843] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:25 2022] DELETE /identity/v3/users/7e2294ae694d4a16aec41f28d6515e06 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:25.483960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:25.485526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:25.498840 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c48832-654c-4c03-8e34-8658244c351a admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:25.555169 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:25.555845 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.556125 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.556390 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] PATH_INFO: `/v3/projects/b3ddce603230418687d10627080ae784` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.557497 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3ddce603230418687d10627080ae784)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:25.559381 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.608842 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3ddce603230418687d10627080ae784', 'request_id': 'req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.615969 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3ddce603230418687d10627080ae784', 'request_id': 'req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.621428 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3ddce603230418687d10627080ae784', 'request_id': 'req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.628438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:25.634120 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] The token cache is being invalidate because project b3ddce603230418687d10627080ae784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:25.634622 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.635687 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-c7e9e10f-b1a6-4918-897d-6779e0be6cdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:25.637213 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1419/2844] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:25 2022] DELETE /identity/v3/projects/b3ddce603230418687d10627080ae784 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:25.642249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.642743 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.642859 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.668587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:25.669875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:25.701386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:25.732916 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1426/2845] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:25.741005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.741287 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.741555 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.761685 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:25.762875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:25.770638 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1420/2846] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:25.774911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:25.784692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6cdaf-3e2a-413a-9e49-c47218d6f1d9 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:25.789359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:25.789961 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.790245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.790524 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.791169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81459) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:25.791479 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:25.793110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.804881 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1427/2847] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:25.808832 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:25.817782 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b585f6d-4981-4a60-9b24-0176b78fc23a None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:25.822856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:25.823435 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.823738 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.824006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.824650 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:25.826511 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.852458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1421/2848] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:25 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:25.856601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:25.867155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411bcea-42be-456f-a4bd-d888156a4dd7 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:25.925418 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:25.926021 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.926296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.926565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.927212 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:25.929661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.953467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1428/2849] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:25 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:25.954649 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:25.966765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526b2282-2aa0-4841-a112-efa023b33e09 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:25.972159 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:25.973011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:25.973295 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:25.973563 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:25.974199 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:25.975807 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:25.985755 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1422/2850] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:25.989911 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.000296 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531c483-9b1e-43e4-baea-21d3d24c3c07 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.004875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11035fca-6a4b-4c0d-8dbe-7c2775f345c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.005471 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11035fca-6a4b-4c0d-8dbe-7c2775f345c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.005750 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11035fca-6a4b-4c0d-8dbe-7c2775f345c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.006667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11035fca-6a4b-4c0d-8dbe-7c2775f345c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.007306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11035fca-6a4b-4c0d-8dbe-7c2775f345c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.008938 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11035fca-6a4b-4c0d-8dbe-7c2775f345c9 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.019809 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1429/2851] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:26.023703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.033539 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46025fff-3532-455a-b8b3-744864662e14 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.038070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.038727 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.039004 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.039275 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] PATH_INFO: `/v3/projects/a3d4cf2f1e7445aea61c9cd77e9e055f/users/e5501dc77a7341509a231e97ab16deae/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.053820 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d4cf2f1e7445aea61c9cd77e9e055f, user_id=e5501dc77a7341509a231e97ab16deae, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.056467 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.069440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1423/2852] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:26 2022] PUT /identity/v3/projects/a3d4cf2f1e7445aea61c9cd77e9e055f/users/e5501dc77a7341509a231e97ab16deae/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:26.080532 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.080821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.081093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.108385 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] MFA Rules not processed for user `e5501dc77a7341509a231e97ab16deae`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:26.109574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:26.173658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1430/2853] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:26.178924 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.239173 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a4b5a0-6831-4b2a-9a5d-e7a022fc380c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.244266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.244852 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.245128 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.245397 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] PATH_INFO: `/v3/projects` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.246040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.247948 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.273112 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1424/2854] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:26 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:26.277414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.290065 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af026bd-3313-430f-bb4c-a18b69843345 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.315893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:26.346458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.347066 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.347347 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.347655 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] PATH_INFO: `/v3/users` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.348292 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.350829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.371597 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1431/2855] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:26 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:26.375672 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.385765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29dc1a-2903-4506-a345-e9f72cf6adcf admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.390291 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.390877 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.391153 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.391423 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.392089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.393697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.403853 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1425/2856] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:26.407703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.417821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2491d3fa-bfa0-441d-a177-05f2eab8303c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.422315 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.422950 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.423227 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.423496 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] PATH_INFO: `/v3/projects/1294844eaa9a4a3aac35bdd479c591bf/users/97123dbca54445a98baadd37b5474c4a/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.437805 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=1294844eaa9a4a3aac35bdd479c591bf, user_id=97123dbca54445a98baadd37b5474c4a, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.440467 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.452779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1432/2857] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:26 2022] PUT /identity/v3/projects/1294844eaa9a4a3aac35bdd479c591bf/users/97123dbca54445a98baadd37b5474c4a/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:13:26.456522 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.466653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad5f2-9710-4a3f-993a-3d7123eb769d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.521131 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0356145-0684-447f-99de-5f267b670c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.522216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356145-0684-447f-99de-5f267b670c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.522496 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356145-0684-447f-99de-5f267b670c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.522762 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0356145-0684-447f-99de-5f267b670c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.523395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356145-0684-447f-99de-5f267b670c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.524991 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0356145-0684-447f-99de-5f267b670c61 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.534433 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1426/2858] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:26.539086 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.549269 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f5156f-ec2f-46f4-bbe2-5c678e2df5ef admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.553816 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.554457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.554735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.555008 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] PATH_INFO: `/v3/projects/1294844eaa9a4a3aac35bdd479c591bf/users/97123dbca54445a98baadd37b5474c4a/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.556944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1294844eaa9a4a3aac35bdd479c591bf, user_id=97123dbca54445a98baadd37b5474c4a, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.559566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.569420 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1433/2859] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:26 2022] PUT /identity/v3/projects/1294844eaa9a4a3aac35bdd479c591bf/users/97123dbca54445a98baadd37b5474c4a/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:26.579185 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.579681 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.579793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.588205 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:26.609257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] MFA Rules not processed for user `97123dbca54445a98baadd37b5474c4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:26.610457 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:26.676999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1427/2860] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:26.689262 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:26.690953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:26.748319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72f025-a34b-4faa-99de-d29f864ca13a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:26.751923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:26.752525 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:26.752804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:26.753079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:26.761922 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:26.763473 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:26.773548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1434/2861] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:27.073476 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:27.082552 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6927b28a-160e-4dcd-a552-c2041cf3088b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:27.087827 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97123dbca54445a98baadd37b5474c4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1294844eaa9a4a3aac35bdd479c591bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:27.088439 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.088724 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.089002 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.089653 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:27.089963 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:27.091718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:27.103646 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1428/2862] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:27.107457 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:27.116421 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c65eba-8b86-4f6b-a83a-26bdfb4c2e02 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:27.120889 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97123dbca54445a98baadd37b5474c4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1294844eaa9a4a3aac35bdd479c591bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:27.121483 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.121766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.122039 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.122670 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:27.124579 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:27.151249 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1435/2863] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:27 2022] POST /identity/v3/projects => generated 355 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:27.159113 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:27.168932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c64b62-58d4-4ab1-9951-a1fe9c444b4d tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:27.224089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97123dbca54445a98baadd37b5474c4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1294844eaa9a4a3aac35bdd479c591bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:27.224740 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.225022 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.225298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.225935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:27.226243 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:27.227837 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:27.239077 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1429/2864] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:27.276414 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:27.286598 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac226b25-f241-4777-b589-e325ec5b5831 tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:27.291666 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '97123dbca54445a98baadd37b5474c4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1294844eaa9a4a3aac35bdd479c591bf', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:27.292574 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.292861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.293171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] PATH_INFO: `/v3/projects/879d3a49e5a54c069b8526129f890d5f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.294977 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorizing `identity:delete_project(project_id=879d3a49e5a54c069b8526129f890d5f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:27.298211 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:27.346042 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879d3a49e5a54c069b8526129f890d5f', 'request_id': 'req-96b55d48-d771-49d9-bf30-7cd75788595b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.352466 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879d3a49e5a54c069b8526129f890d5f', 'request_id': 'req-96b55d48-d771-49d9-bf30-7cd75788595b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.353693 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=88,keystone:INSERT=7,keystone:DELETE=5 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:27.360533 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879d3a49e5a54c069b8526129f890d5f', 'request_id': 'req-96b55d48-d771-49d9-bf30-7cd75788595b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.371894 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] The token cache is being invalidate because project 879d3a49e5a54c069b8526129f890d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:27.372182 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.372851 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-96b55d48-d771-49d9-bf30-7cd75788595b tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.374327 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1436/2865] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:27 2022] DELETE /identity/v3/projects/879d3a49e5a54c069b8526129f890d5f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:27.559325 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:27.561071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:27.625671 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff6e5b-f2b0-45da-b1b5-f588c80a22dc tempest-VolumeQuotaClassesTest-124304396 tempest-VolumeQuotaClassesTest-124304396-project-admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:27.637835 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:27.681081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:27.681712 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.681989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.682260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] PATH_INFO: `/v3/users/e5501dc77a7341509a231e97ab16deae` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.683387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5501dc77a7341509a231e97ab16deae)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:27.685334 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:27.729998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5501dc77a7341509a231e97ab16deae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.736803 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5501dc77a7341509a231e97ab16deae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.737560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5501dc77a7341509a231e97ab16deae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.750455 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5501dc77a7341509a231e97ab16deae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.751095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5501dc77a7341509a231e97ab16deae'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:27.763834 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1430/2866] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:27 2022] DELETE /identity/v3/users/e5501dc77a7341509a231e97ab16deae => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:27.768202 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.768686 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.768811 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.787758 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:27.789012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:27.845888 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1437/2867] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:27.855823 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:27.857202 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:27.914034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94317ed-3706-4c00-9ccb-8383289e31a6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:27.917628 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:27.918216 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:27.918491 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:27.918758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:27.927564 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:27.929598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:27.938798 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1431/2868] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:28.203428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:28.205006 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:28.217781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df33dca2-cf96-4403-aab3-bb3151914e01 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:28.222565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:28.223204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:28.223485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:28.223808 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] PATH_INFO: `/v3/projects/a3d4cf2f1e7445aea61c9cd77e9e055f` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:28.225528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d4cf2f1e7445aea61c9cd77e9e055f)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:28.227568 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:28.275696 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d4cf2f1e7445aea61c9cd77e9e055f', 'request_id': 'req-82459ee0-8681-48e5-a89b-67997e0b6142'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.282775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d4cf2f1e7445aea61c9cd77e9e055f', 'request_id': 'req-82459ee0-8681-48e5-a89b-67997e0b6142'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.289493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d4cf2f1e7445aea61c9cd77e9e055f', 'request_id': 'req-82459ee0-8681-48e5-a89b-67997e0b6142'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.299924 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] The token cache is being invalidate because project a3d4cf2f1e7445aea61c9cd77e9e055f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:28.300229 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.300862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.302310 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1438/2869] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:28 2022] DELETE /identity/v3/projects/a3d4cf2f1e7445aea61c9cd77e9e055f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:28.306219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:28.315742 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf8a7e-db50-490f-8758-e8d9fe638bfe service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:28.353945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:28.373849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:28.374477 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:28.374756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:28.375025 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] PATH_INFO: `/v3/users/97123dbca54445a98baadd37b5474c4a` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:28.376190 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97123dbca54445a98baadd37b5474c4a)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:28.378096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:28.423095 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97123dbca54445a98baadd37b5474c4a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.429486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97123dbca54445a98baadd37b5474c4a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.430196 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97123dbca54445a98baadd37b5474c4a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.442141 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97123dbca54445a98baadd37b5474c4a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.442779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3ab3fe24-1c3e-480a-aa52-a72328c6c390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97123dbca54445a98baadd37b5474c4a'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.455320 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1432/2870] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:28 2022] DELETE /identity/v3/users/97123dbca54445a98baadd37b5474c4a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:28.720306 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:28.721765 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:28.739144 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82459ee0-8681-48e5-a89b-67997e0b6142 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:28.796730 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:28.797354 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:28.797620 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:28.797893 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] PATH_INFO: `/v3/projects/1294844eaa9a4a3aac35bdd479c591bf` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:28.798996 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1294844eaa9a4a3aac35bdd479c591bf)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:28.800942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:28.811684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:28.848736 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1294844eaa9a4a3aac35bdd479c591bf', 'request_id': 'req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.855587 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1294844eaa9a4a3aac35bdd479c591bf', 'request_id': 'req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.861706 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1294844eaa9a4a3aac35bdd479c591bf', 'request_id': 'req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.872364 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] The token cache is being invalidate because project 1294844eaa9a4a3aac35bdd479c591bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:28.872652 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.873294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-4ea855de-10a0-4d6e-9c18-3719cd4bac7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:28.874964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1439/2871] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:28 2022] DELETE /identity/v3/projects/1294844eaa9a4a3aac35bdd479c591bf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:28.879897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:28.880369 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:28.880473 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:28.900697 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:28.901919 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:28.959717 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1433/2872] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:28.967311 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:28.967645 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:28.967907 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:28.987218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:28.988446 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:28.996703 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1440/2873] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:29.000713 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.010690 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f935b553-b174-489e-9d2e-d74e2bf5b909 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.015150 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.015792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.016068 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.016335 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.016971 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:29.017268 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.018822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.030018 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1434/2874] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:29.034245 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.043198 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a856ab83-320f-400c-ac69-d0dee006bb50 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.047791 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.048378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.048654 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.048923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.049549 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.051426 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.078527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1441/2875] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:29.082570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.092964 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e2522b-4830-43df-b5fb-5565481a70aa admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.135968 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:29.149990 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.150584 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.150863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.151134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.151792 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.153693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.174601 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1435/2876] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:29.179167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.189566 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2c37fe-2541-433e-9646-c8f97bcf6e2d admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.194152 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.194747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.195030 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.195301 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.195983 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.197571 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.207662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1442/2877] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:29.211300 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.221289 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55055eba-52dd-4d22-9ee4-b75cc0b01241 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.226353 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97327faf-63e1-40be-8267-ca3afc87247e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.226932 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97327faf-63e1-40be-8267-ca3afc87247e admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.227209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97327faf-63e1-40be-8267-ca3afc87247e admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.227478 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97327faf-63e1-40be-8267-ca3afc87247e admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.228138 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97327faf-63e1-40be-8267-ca3afc87247e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.229696 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97327faf-63e1-40be-8267-ca3afc87247e admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.239904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1436/2878] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:29.244257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.254098 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdbf737-8bbe-4d5f-9e1e-2c079ec36cb0 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.258667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.259339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.259651 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.259910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] PATH_INFO: `/v3/projects/6e9a3df5102b47e39a8c3340e0deac9d/users/408e8ee5c65342d0aa05364885336afa/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.274191 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a3df5102b47e39a8c3340e0deac9d, user_id=408e8ee5c65342d0aa05364885336afa, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.276855 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.289454 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1443/2879] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:29 2022] PUT /identity/v3/projects/6e9a3df5102b47e39a8c3340e0deac9d/users/408e8ee5c65342d0aa05364885336afa/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:29.300730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.301010 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.301278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.328371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] MFA Rules not processed for user `408e8ee5c65342d0aa05364885336afa`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:29.329607 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:29.391654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1437/2880] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 07 16:13:29.396153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.456366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f1250-57ba-458f-bdd6-b68ec03fa682 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.461487 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.462083 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.462361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.462632 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.463278 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.465184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.493115 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1444/2881] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 07 16:13:29.497108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.507774 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60a8e2-517b-4b31-80ab-21515c59c7fe None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.562955 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.563654 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.564093 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.564412 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.565197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.567086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.588253 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1438/2882] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:29.592206 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.604646 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f151a-081d-4ff7-b8bf-8130c86c92e6 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.609838 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.610435 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.610715 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.610985 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.611661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.613255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.623901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1445/2883] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:29.627942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.637946 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1112fb5-65fa-4dd4-817b-0fd22b284b5c admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.642619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29263df-6df6-481a-89ea-e7f5c7d10377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.643211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29263df-6df6-481a-89ea-e7f5c7d10377 admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.643493 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29263df-6df6-481a-89ea-e7f5c7d10377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.643802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29263df-6df6-481a-89ea-e7f5c7d10377 admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.644440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29263df-6df6-481a-89ea-e7f5c7d10377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.646048 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29263df-6df6-481a-89ea-e7f5c7d10377 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.656304 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1439/2884] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:29.660605 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.670923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d94c4b-b50b-4901-8006-24d640af611a admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.675506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.676171 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.676453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.676727 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] PATH_INFO: `/v3/projects/e6bc313551b9446e8aafec7e2e44186e/users/3aa5234fb0334c86aecea8ee2ac3ba86/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.679794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:29.694268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6bc313551b9446e8aafec7e2e44186e, user_id=3aa5234fb0334c86aecea8ee2ac3ba86, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.697513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.711093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1446/2885] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:29 2022] PUT /identity/v3/projects/e6bc313551b9446e8aafec7e2e44186e/users/3aa5234fb0334c86aecea8ee2ac3ba86/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:29.721490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.721881 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.722161 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.750321 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] MFA Rules not processed for user `3aa5234fb0334c86aecea8ee2ac3ba86`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:29.751580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:29.772023 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:29.816422 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1440/2886] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:29.821491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.882901 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f79f6-1228-4770-ad4b-555be325e31e admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.887458 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.888079 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.888359 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.888633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.889261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.891140 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:29.918386 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1447/2887] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:29.922447 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:29.933194 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7a58df-a222-437b-a9f4-3487d4ed32f3 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:29.988758 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:29.989363 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:29.989643 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:29.989917 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:29.990579 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:29.992531 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.013626 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1441/2888] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:29 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:30.017513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:30.028169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daab10b-6ff9-47ee-bc43-dfbfb8dbca87 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:30.032658 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:30.033250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.033528 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.033798 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.034425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:30.036099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.046431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1448/2889] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:30.050223 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:30.060001 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6b9a96-1def-4028-88e6-5bed4c44f7f3 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:30.064603 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:30.065239 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.065520 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.065793 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] PATH_INFO: `/v3/projects/4c7bfcabbb684032977b4e6d6188c356/users/4c3a3430dd87422abac4bb16be09769d/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.080485 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7bfcabbb684032977b4e6d6188c356, user_id=4c3a3430dd87422abac4bb16be09769d, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:30.083096 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.096489 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1442/2890] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:30 2022] PUT /identity/v3/projects/4c7bfcabbb684032977b4e6d6188c356/users/4c3a3430dd87422abac4bb16be09769d/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:30.100515 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:30.110663 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a597582-c27c-4510-b01e-0a034e6304c3 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:30.170909 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961437f-c686-4875-8654-aed33df2ebfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:30.171516 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961437f-c686-4875-8654-aed33df2ebfa admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.171827 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961437f-c686-4875-8654-aed33df2ebfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.172099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961437f-c686-4875-8654-aed33df2ebfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.172745 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961437f-c686-4875-8654-aed33df2ebfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:30.174369 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961437f-c686-4875-8654-aed33df2ebfa admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.184553 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1449/2891] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:30.188855 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:30.199574 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d28341-9370-42af-8f78-be642fcf0b4d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:30.204248 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:30.204897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.205172 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.205438 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] PATH_INFO: `/v3/projects/4c7bfcabbb684032977b4e6d6188c356/users/4c3a3430dd87422abac4bb16be09769d/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.207379 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7bfcabbb684032977b4e6d6188c356, user_id=4c3a3430dd87422abac4bb16be09769d, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:30.210053 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.220440 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1443/2892] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:30 2022] PUT /identity/v3/projects/4c7bfcabbb684032977b4e6d6188c356/users/4c3a3430dd87422abac4bb16be09769d/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:13:30.230512 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.230806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.231082 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.260399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] MFA Rules not processed for user `4c3a3430dd87422abac4bb16be09769d`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:30.261610 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:30.303912 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:30.329657 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1450/2893] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:30.341318 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:30.343016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:30.401411 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d718183-140f-456d-9f8e-88eb3aefdac6 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:30.405089 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:30.405678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.405958 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.406226 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.415136 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:30.416537 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.425802 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1444/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:30.467190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:30.468600 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:30.480575 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d6cd5-8b03-4241-bdcf-501bb35a4005 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:30.484155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae0fe0-aa3b-48ca-a876-094997ae3bed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:30.484747 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0fe0-aa3b-48ca-a876-094997ae3bed service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:30.485024 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0fe0-aa3b-48ca-a876-094997ae3bed service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:30.485294 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae0fe0-aa3b-48ca-a876-094997ae3bed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:30.539951 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae0fe0-aa3b-48ca-a876-094997ae3bed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:30.541444 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bae0fe0-aa3b-48ca-a876-094997ae3bed service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:30.550059 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1451/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:33.418149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:33.437280 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:33.480342 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7203556-48d5-4988-bc2c-b1e7afb26a30 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:33.484833 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:33.485451 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:33.485729 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:33.486000 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] PATH_INFO: `/v3/users/408e8ee5c65342d0aa05364885336afa` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:33.487110 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] RBAC: Authorizing `identity:delete_user(user_id=408e8ee5c65342d0aa05364885336afa)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:33.489035 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:33.533378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '408e8ee5c65342d0aa05364885336afa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:33.541142 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '408e8ee5c65342d0aa05364885336afa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:33.541779 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '408e8ee5c65342d0aa05364885336afa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:33.554047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '408e8ee5c65342d0aa05364885336afa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:33.554731 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '408e8ee5c65342d0aa05364885336afa'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:33.566967 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1445/2896] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:33 2022] DELETE /identity/v3/users/408e8ee5c65342d0aa05364885336afa => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:33.571428 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:33.571944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:33.572070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:33.590452 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:33.591673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:33.661674 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1452/2897] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:33.671720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:33.673123 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:33.732490 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cfcba8-f2aa-4d94-b2c2-6c68fae7151d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:33.736081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:33.736684 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:33.736960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:33.737227 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:33.746556 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:33.748013 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:33.757211 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1446/2898] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:34.015218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:34.016680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:34.029506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6a73e-79af-496b-835b-c688848c48ad None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:34.034399 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:34.035036 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:34.035316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:34.035633 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] PATH_INFO: `/v3/projects/6e9a3df5102b47e39a8c3340e0deac9d` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:34.036873 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9a3df5102b47e39a8c3340e0deac9d)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:34.039352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:34.086337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9a3df5102b47e39a8c3340e0deac9d', 'request_id': 'req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.093406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9a3df5102b47e39a8c3340e0deac9d', 'request_id': 'req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.099903 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9a3df5102b47e39a8c3340e0deac9d', 'request_id': 'req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.110478 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] The token cache is being invalidate because project 6e9a3df5102b47e39a8c3340e0deac9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:34.110763 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.111378 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.112861 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1453/2899] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:34 2022] DELETE /identity/v3/projects/6e9a3df5102b47e39a8c3340e0deac9d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:34.116915 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:34.126470 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f688087-ca8f-496c-a758-6f26fe56ef03 service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:34.164257 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:34.187677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:34.188309 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:34.188591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:34.188864 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] PATH_INFO: `/v3/users/3aa5234fb0334c86aecea8ee2ac3ba86` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:34.190027 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa5234fb0334c86aecea8ee2ac3ba86)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:34.191929 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:34.237610 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa5234fb0334c86aecea8ee2ac3ba86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.244630 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa5234fb0334c86aecea8ee2ac3ba86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.245395 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa5234fb0334c86aecea8ee2ac3ba86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.257538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa5234fb0334c86aecea8ee2ac3ba86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.258175 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa5234fb0334c86aecea8ee2ac3ba86'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.270538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1447/2900] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:34 2022] DELETE /identity/v3/users/3aa5234fb0334c86aecea8ee2ac3ba86 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:34.534313 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:34.535775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:34.545037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:34.554199 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc48a13-12a3-44a0-af75-4e8c2c8afb53 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:34.613552 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:34.614222 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:34.614506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:34.614781 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] PATH_INFO: `/v3/projects/e6bc313551b9446e8aafec7e2e44186e` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:34.615940 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6bc313551b9446e8aafec7e2e44186e)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:34.617882 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:34.663319 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6bc313551b9446e8aafec7e2e44186e', 'request_id': 'req-0635dc45-0004-4a0f-b35a-b3f23ded2406'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.670340 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6bc313551b9446e8aafec7e2e44186e', 'request_id': 'req-0635dc45-0004-4a0f-b35a-b3f23ded2406'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.676522 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6bc313551b9446e8aafec7e2e44186e', 'request_id': 'req-0635dc45-0004-4a0f-b35a-b3f23ded2406'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.687117 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] The token cache is being invalidate because project e6bc313551b9446e8aafec7e2e44186e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:34.687403 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.688060 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.689514 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1454/2901] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:34 2022] DELETE /identity/v3/projects/e6bc313551b9446e8aafec7e2e44186e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:34.693674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:34.696055 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:34.709139 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893af25c-f450-46f0-b66e-31733a8ec3c5 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:34.765448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:34.766040 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:34.766323 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:34.766598 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] PATH_INFO: `/v3/users/4c3a3430dd87422abac4bb16be09769d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:34.767756 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c3a3430dd87422abac4bb16be09769d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:34.769677 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:34.813733 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c3a3430dd87422abac4bb16be09769d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.821312 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c3a3430dd87422abac4bb16be09769d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.821936 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c3a3430dd87422abac4bb16be09769d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.827263 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:34.834682 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c3a3430dd87422abac4bb16be09769d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.835378 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-093be11d-ed86-49d1-b150-7371d910a67c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c3a3430dd87422abac4bb16be09769d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:34.851663 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1448/2902] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:34 2022] DELETE /identity/v3/users/4c3a3430dd87422abac4bb16be09769d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:35.121728 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.123236 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:35.142080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0635dc45-0004-4a0f-b35a-b3f23ded2406 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.206492 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.207111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.207394 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.207716 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] PATH_INFO: `/v3/projects/4c7bfcabbb684032977b4e6d6188c356` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.208833 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7bfcabbb684032977b4e6d6188c356)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.210775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.255216 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:35.271447 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7bfcabbb684032977b4e6d6188c356', 'request_id': 'req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:35.278949 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7bfcabbb684032977b4e6d6188c356', 'request_id': 'req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:35.285662 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7bfcabbb684032977b4e6d6188c356', 'request_id': 'req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:35.301944 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] The token cache is being invalidate because project 4c7bfcabbb684032977b4e6d6188c356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:35.302235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:35.302953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-c92cbeb9-ce57-4be7-bcf2-93ff587418d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:35.305106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1455/2903] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:35 2022] DELETE /identity/v3/projects/4c7bfcabbb684032977b4e6d6188c356 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:35.311042 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.311398 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.311715 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.332787 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:35.333998 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:35.395221 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1449/2904] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:35.404186 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.404470 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.404740 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.426489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:35.427735 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:35.435697 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1456/2905] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:35.440047 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.450149 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0c1eeb-94ab-46fb-914a-5885aa6583b6 None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.454909 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.455507 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.455812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.456081 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.456720 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:35.457021 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.458718 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.472306 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1450/2906] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:35.477274 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.486964 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bf26b-88b6-4f77-af2c-12035ab22e88 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.491700 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.492328 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.492591 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.492864 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.493493 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.495392 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.523644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1457/2907] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:35 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:35.528006 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.542632 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a707b5a-32f3-464e-91e1-a1029973eeed admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.599600 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.600121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.600953 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.601236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.602019 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.605236 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.628388 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1451/2908] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:35 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:35.632769 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.644405 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e856f-a0b5-44c7-aafb-7a66485e7624 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.651111 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.652219 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.652714 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.653218 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.654395 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.656160 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.667431 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1458/2909] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:35.672875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.682070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a16c52-7d44-48b9-9375-ab64394e06df admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.687703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dcedf0-0db2-459e-93b9-0daacfc174bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.688705 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcedf0-0db2-459e-93b9-0daacfc174bb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.689167 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcedf0-0db2-459e-93b9-0daacfc174bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.689658 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dcedf0-0db2-459e-93b9-0daacfc174bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.690392 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcedf0-0db2-459e-93b9-0daacfc174bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.692673 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dcedf0-0db2-459e-93b9-0daacfc174bb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.705242 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1452/2910] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:35.709429 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.720441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e702b2-8f52-4635-b028-e656adc24c10 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.725074 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.726361 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.726647 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.726923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] PATH_INFO: `/v3/projects/bb830ae65c1244f7a77f2336ee84671b/users/fb99ee63aedc45b2896d41fda1b13670/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.742441 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb830ae65c1244f7a77f2336ee84671b, user_id=fb99ee63aedc45b2896d41fda1b13670, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.745136 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.757453 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1459/2911] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:35 2022] PUT /identity/v3/projects/bb830ae65c1244f7a77f2336ee84671b/users/fb99ee63aedc45b2896d41fda1b13670/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:35.762399 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac559e-276c-453b-90eb-f526b74a82fe None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.762765 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac559e-276c-453b-90eb-f526b74a82fe None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.763109 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac559e-276c-453b-90eb-f526b74a82fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.782488 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-dbac559e-276c-453b-90eb-f526b74a82fe None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:35.783732 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-dbac559e-276c-453b-90eb-f526b74a82fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:35.823631 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:35.843875 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1453/2912] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:35.853636 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:35.855240 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:35.911604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a83f5d1-e3b7-46c8-8d4c-e077d63ac01b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:35.915256 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:35.915890 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:35.916230 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:35.916506 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:35.925374 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:35.926829 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:35.936190 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1460/2913] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:36.540016 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.540302 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.540572 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.569214 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] MFA Rules not processed for user `fb99ee63aedc45b2896d41fda1b13670`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:36.570434 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:36.636340 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1454/2914] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:36.640862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:36.650601 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4aea13-855c-4b95-a169-e0d16d2c5136 service neutron] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:36.655155 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:36.655830 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.656106 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.656377 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.657167 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:36.659090 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:36.689257 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1461/2915] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:36 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:36.693448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:36.705942 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43ae0d9-4014-4ed0-be87-8190eb9ca81b None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:36.766770 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:36.767588 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.767989 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.768162 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.769015 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:36.772204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:36.797090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1455/2916] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:36 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:36.798268 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:36.809183 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f5200a-e8e1-41b4-9a93-c1921564f214 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:36.814067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:36.814726 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.815013 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.815287 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.815970 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:36.817578 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:36.828684 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1462/2917] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 07 16:13:36.832868 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:36.844311 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8b102-dd52-42ae-a9d0-36f4687202bd admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:36.849328 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:36.849975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.850944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.851220 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] PATH_INFO: `/v3/projects/58d612928e914dea89fcf234e234c400/users/6c6108eac6e44270b61b73ad618cefc4/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.865822 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d612928e914dea89fcf234e234c400, user_id=6c6108eac6e44270b61b73ad618cefc4, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:36.869350 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:36.882623 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1456/2918] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:36 2022] PUT /identity/v3/projects/58d612928e914dea89fcf234e234c400/users/6c6108eac6e44270b61b73ad618cefc4/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:13:36.886489 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:36.896540 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08edb12-c9c1-4884-8a07-a8c4cf21fd69 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:36.922852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:36.954247 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0bfbc7-98b7-4c4f-8f09-f71c7351ccd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:36.954794 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bfbc7-98b7-4c4f-8f09-f71c7351ccd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.955080 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bfbc7-98b7-4c4f-8f09-f71c7351ccd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.955356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0bfbc7-98b7-4c4f-8f09-f71c7351ccd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.956113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bfbc7-98b7-4c4f-8f09-f71c7351ccd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:36.957709 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0bfbc7-98b7-4c4f-8f09-f71c7351ccd8 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:36.967822 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1463/2919] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:36.972360 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:36.983252 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8a4b3-a40c-4bd8-bf5c-d4651d39e971 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:36.987925 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:36.988570 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:36.988849 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:36.989121 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] PATH_INFO: `/v3/projects/58d612928e914dea89fcf234e234c400/users/6c6108eac6e44270b61b73ad618cefc4/roles/047a84bbadcb4ccf9187faec9f7853fc` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:36.991036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d612928e914dea89fcf234e234c400, user_id=6c6108eac6e44270b61b73ad618cefc4, role_id=047a84bbadcb4ccf9187faec9f7853fc)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:36.993683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:37.003683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1457/2920] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:36 2022] PUT /identity/v3/projects/58d612928e914dea89fcf234e234c400/users/6c6108eac6e44270b61b73ad618cefc4/roles/047a84bbadcb4ccf9187faec9f7853fc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 07 16:13:37.619416 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:37.619835 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:37.620175 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:37.651517 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] MFA Rules not processed for user `6c6108eac6e44270b61b73ad618cefc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:37.652862 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:37.725057 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1464/2921] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:37 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:37.736449 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:37.738189 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:37.771274 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:37.800317 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9fdce-c4e8-4065-ad5d-9065684a99b2 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:37.804466 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:37.805064 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:37.805338 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:37.805605 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:37.861704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:37.863145 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:37.872502 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1458/2922] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:39.391250 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:39.392844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:39.405425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc86f619-5682-4e86-a318-661ce3dc4cd7 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:39.409012 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:39.409664 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:39.409898 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:39.410172 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:39.419625 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:39.421047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:39.430878 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1465/2923] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:13:47.868769 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=36 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:49.426254 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=56 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:52.359032 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:52.424620 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86154f-41dc-4c57-8bb1-56460b023087 service cinder] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:52.429080 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:52.429698 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:52.429975 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:52.430873 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] PATH_INFO: `/v3/users/fb99ee63aedc45b2896d41fda1b13670` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:52.432005 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb99ee63aedc45b2896d41fda1b13670)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:52.433904 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:52.479204 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb99ee63aedc45b2896d41fda1b13670'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.486057 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb99ee63aedc45b2896d41fda1b13670'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.486775 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb99ee63aedc45b2896d41fda1b13670'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.499524 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb99ee63aedc45b2896d41fda1b13670'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.500187 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb99ee63aedc45b2896d41fda1b13670'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.512036 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1459/2924] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:52 2022] DELETE /identity/v3/users/fb99ee63aedc45b2896d41fda1b13670 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:52.681521 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:52.696952 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8153f15-ff72-4bac-b3fa-dee7ffc2ea19 service cinder] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:52.756870 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:52.757424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:52.757704 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:52.757979 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] PATH_INFO: `/v3/projects/bb830ae65c1244f7a77f2336ee84671b` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:52.759110 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb830ae65c1244f7a77f2336ee84671b)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:52.761037 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:52.805424 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb830ae65c1244f7a77f2336ee84671b', 'request_id': 'req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.812204 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb830ae65c1244f7a77f2336ee84671b', 'request_id': 'req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.818259 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb830ae65c1244f7a77f2336ee84671b', 'request_id': 'req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.828383 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] The token cache is being invalidate because project bb830ae65c1244f7a77f2336ee84671b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:52.828667 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.829289 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.830734 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1466/2925] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:52 2022] DELETE /identity/v3/projects/bb830ae65c1244f7a77f2336ee84671b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:52.834839 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:52.836766 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:52.849985 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c04456-bc46-453a-b4f3-7a397c4fb879 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:52.906193 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:52.906819 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:52.907098 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:52.907368 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] PATH_INFO: `/v3/users/6c6108eac6e44270b61b73ad618cefc4` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:52.908526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c6108eac6e44270b61b73ad618cefc4)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:52.910523 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:52.932565 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:52.957014 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c6108eac6e44270b61b73ad618cefc4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.964210 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c6108eac6e44270b61b73ad618cefc4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.964889 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c6108eac6e44270b61b73ad618cefc4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.977573 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c6108eac6e44270b61b73ad618cefc4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.978206 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-ba728d31-2261-481c-8717-d84250caf81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c6108eac6e44270b61b73ad618cefc4'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:52.989920 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1460/2926] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:13:52 2022] DELETE /identity/v3/users/6c6108eac6e44270b61b73ad618cefc4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:53.162293 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.163801 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:53.177228 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798d7719-5055-4e17-ae32-3e6e2dbb8f9b admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.234628 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.235261 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.235569 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.235844 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] PATH_INFO: `/v3/projects/58d612928e914dea89fcf234e234c400` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.236953 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d612928e914dea89fcf234e234c400)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.238853 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.272942 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:53.287589 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d612928e914dea89fcf234e234c400', 'request_id': 'req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:53.295243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d612928e914dea89fcf234e234c400', 'request_id': 'req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:53.301366 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d612928e914dea89fcf234e234c400', 'request_id': 'req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:53.312761 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] The token cache is being invalidate because project 58d612928e914dea89fcf234e234c400 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:13:53.313047 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:53.313680 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-6dc59e5e-4156-4db6-b80c-81d31b97ce6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:13:53.315184 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1467/2927] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:13:53 2022] DELETE /identity/v3/projects/58d612928e914dea89fcf234e234c400 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:53.320260 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.320548 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.320818 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.340480 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:53.341678 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:53.399526 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1461/2928] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:53.407367 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.407692 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.407975 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.427923 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:53.429123 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:53.436918 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1468/2929] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:53.441140 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.450878 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff976f88-d219-4425-9ce6-8e8483c2f25c None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.455389 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.456011 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.456286 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.456581 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.457219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81458) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 07 16:13:53.457538 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.459209 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.471675 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1462/2930] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Sep 7 16:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 07 16:13:53.475843 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.485767 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b322e9-4e72-4322-b41d-17a85dabc049 None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.490491 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.491093 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.491406 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.491722 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] PATH_INFO: `/v3/projects` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.492352 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.494264 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.523349 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1469/2931] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Sep 7 16:13:53 2022] POST /identity/v3/projects => generated 397 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:53.525178 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.534957 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f36d2-a513-40ea-b3a9-397d32191dbb admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.589986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.595481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.595481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.595481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.595481 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.595935 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.616987 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1463/2932] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Sep 7 16:13:53 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 07 16:13:53.621113 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.631434 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a625eb5-95a7-4e6a-b477-f763de39c689 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.636181 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.636779 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.637061 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.637337 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.638157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.639592 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.650169 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1470/2933] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:53.654704 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.667219 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c75c2-31bb-4f85-9aa2-78b419ec0bd7 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.673536 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffff5493-cbe7-4388-8981-b02908db7c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.674134 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff5493-cbe7-4388-8981-b02908db7c7b admin admin] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.674416 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff5493-cbe7-4388-8981-b02908db7c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.674693 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffff5493-cbe7-4388-8981-b02908db7c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.675336 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff5493-cbe7-4388-8981-b02908db7c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.677070 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffff5493-cbe7-4388-8981-b02908db7c7b admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.689067 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1464/2934] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Sep 7 16:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 07 16:13:53.693548 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.703895 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b127982-0fdf-41a5-af39-9c4f2c0afd14 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.708459 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.709099 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.709384 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.709661 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] PATH_INFO: `/v3/projects/3b0d4a903fa34d148812410710212d92/users/d81d01ce682842109c1d8809a73d5bf7/roles/09d9b803d29e48b28dfc0fcb6d976e31` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.725821 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b0d4a903fa34d148812410710212d92, user_id=d81d01ce682842109c1d8809a73d5bf7, role_id=09d9b803d29e48b28dfc0fcb6d976e31)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.729135 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.741243 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1471/2935] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Sep 7 16:13:53 2022] PUT /identity/v3/projects/3b0d4a903fa34d148812410710212d92/users/d81d01ce682842109c1d8809a73d5bf7/roles/09d9b803d29e48b28dfc0fcb6d976e31 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:13:53.752722 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] REQUEST_METHOD: `POST` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.753034 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.753303 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.761814 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:13:53.782703 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.auth.core [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] MFA Rules not processed for user `d81d01ce682842109c1d8809a73d5bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81458) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:13:53.783945 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:53.846500 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1465/2936] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:13:53 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:13:53.857641 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:13:53.859330 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:13:53.914753 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52a728-4dd5-4b1b-9a87-c03e99acdd00 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:13:53.918362 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0623f28-ed56-4b81-8c5f-bb8709c38e5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '396fc6e85a65437898bc9d964e429de3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:13:53.918955 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623f28-ed56-4b81-8c5f-bb8709c38e5e service cinder] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:13:53.919235 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623f28-ed56-4b81-8c5f-bb8709c38e5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:13:53.919507 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0623f28-ed56-4b81-8c5f-bb8709c38e5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:13:53.927644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0623f28-ed56-4b81-8c5f-bb8709c38e5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:13:53.929070 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0623f28-ed56-4b81-8c5f-bb8709c38e5e service cinder] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:13:53.937966 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1472/2937] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:13:53 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:01.992604 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:02.054030 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97943102-f8a1-4ce5-bc53-238675f1218d None None] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:02.058458 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:02.059082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:02.059362 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:02.059674 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] PATH_INFO: `/v3/users/d81d01ce682842109c1d8809a73d5bf7` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:02.060788 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81d01ce682842109c1d8809a73d5bf7)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:02.062730 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:02.108343 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81d01ce682842109c1d8809a73d5bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.115453 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81d01ce682842109c1d8809a73d5bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.116108 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81d01ce682842109c1d8809a73d5bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.127777 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81d01ce682842109c1d8809a73d5bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.128401 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81d01ce682842109c1d8809a73d5bf7'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.141050 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1466/2938] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:14:01 2022] DELETE /identity/v3/users/d81d01ce682842109c1d8809a73d5bf7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:14:02.145565 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] REQUEST_METHOD: `POST` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:02.146032 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:02.146139 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:02.165644 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.auth.core [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] MFA Rules not processed for user `91fe14f693174ef291042aa145475ad4`. Rule list: `[]` (Enabled: `True`). {{(pid=81459) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 07 16:14:02.166850 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:02.218116 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:02.228456 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1473/2939] 10.222.0.17 () {58 vars in 958 bytes} [Wed Sep 7 16:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 07 16:14:02.237858 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:02.239259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:02.253711 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:02.298863 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b55800-c7b6-4731-8f79-096ac4cd4f77 admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:02.302664 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:02.303270 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:02.303577 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:02.303856 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:02.313158 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:02.314583 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:02.324813 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1467/2940] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:02.580947 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:02.582527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:02.597920 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eb5c56-cb61-447a-b038-9f306956670b None None] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:02.602760 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:02.603425 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:02.603751 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:02.604128 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] PATH_INFO: `/v3/projects/3b0d4a903fa34d148812410710212d92` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:02.605153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b0d4a903fa34d148812410710212d92)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:02.607153 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:02.653527 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b0d4a903fa34d148812410710212d92', 'request_id': 'req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.659795 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b0d4a903fa34d148812410710212d92', 'request_id': 'req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.666107 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b0d4a903fa34d148812410710212d92', 'request_id': 'req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.676316 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] The token cache is being invalidate because project 3b0d4a903fa34d148812410710212d92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81459) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:14:02.676604 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.677223 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:02.678659 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1474/2941] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:14:02 2022] DELETE /identity/v3/projects/3b0d4a903fa34d148812410710212d92 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:14:05.446829 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:05.455486 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da7087b-5168-48c6-9f6f-b172d703452f service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:05.503319 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:05.503960 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:05.504244 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:05.504515 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:05.506298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:05.565282 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:05.566708 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:05.576144 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1468/2942] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:06.627537 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:06.628910 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:06.711255 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111dfeb1-5c39-4fe0-8c63-8cada1d92f2c admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:06.717639 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a9090ab0ffa4b7eafcc742880237229', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:06.718258 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:06.718546 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:06.718825 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:06.720904 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:06.737622 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:06.740033 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:06.753318 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1475/2943] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:10.458619 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:10.460111 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:10.471082 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d049cb9d-979b-4cc8-9414-3758a2fc5c2c service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:10.474683 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '690506f288f84debaadb4c8975759dad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:10.475278 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] REQUEST_METHOD: `GET` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:10.475575 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:10.475865 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:10.478033 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:10.539448 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:10.541090 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:10.551086 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1469/2944] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:14:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:14:16.527845 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:16.529339 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:16.608475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48855a5-5c52-46ca-a333-ab2ef31fe4d5 service nova] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:16.613246 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:16.613875 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:16.614157 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:16.614433 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] PATH_INFO: `/v3/users/22aa615f2c7c4c6eaa694f551e38436a` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:16.615572 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22aa615f2c7c4c6eaa694f551e38436a)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:16.618048 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:16.619518 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:16.668224 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22aa615f2c7c4c6eaa694f551e38436a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:16.675485 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22aa615f2c7c4c6eaa694f551e38436a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:16.687828 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22aa615f2c7c4c6eaa694f551e38436a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:16.688481 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22aa615f2c7c4c6eaa694f551e38436a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:16.689089 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22aa615f2c7c4c6eaa694f551e38436a'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:16.701879 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1476/2945] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:14:16 2022] DELETE /identity/v3/users/22aa615f2c7c4c6eaa694f551e38436a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:14:16.874560 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:16.878760 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=100 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:16.935484 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4aa77d-c84e-4a68-9202-15ee2d3921ae service neutron] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:16.996916 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:16.997585 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:16.997986 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:16.998266 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] PATH_INFO: `/v3/projects/266789e160554ec0a0a334304a0ad39d` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:16.999413 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] RBAC: Authorizing `identity:delete_project(project_id=266789e160554ec0a0a334304a0ad39d)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:17.001371 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:17.049944 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '266789e160554ec0a0a334304a0ad39d', 'request_id': 'req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.059259 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '266789e160554ec0a0a334304a0ad39d', 'request_id': 'req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.066357 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '266789e160554ec0a0a334304a0ad39d', 'request_id': 'req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.078091 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] The token cache is being invalidate because project 266789e160554ec0a0a334304a0ad39d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:14:17.078387 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.079118 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.083763 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1470/2946] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:14:16 2022] DELETE /identity/v3/projects/266789e160554ec0a0a334304a0ad39d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:14:17.084775 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:17.086215 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:17.099852 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4698e-18b1-4d7b-bc12-2c5935a061a4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:17.158968 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:17.159648 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:17.159935 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:17.160208 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] PATH_INFO: `/v3/users/690176f79d9d4df0ae1a4078428b1375` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:17.161356 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=690176f79d9d4df0ae1a4078428b1375)` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:17.163291 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:17.210738 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690176f79d9d4df0ae1a4078428b1375'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.217762 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690176f79d9d4df0ae1a4078428b1375'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.230494 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690176f79d9d4df0ae1a4078428b1375'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.231173 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690176f79d9d4df0ae1a4078428b1375'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.231806 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.notifications [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690176f79d9d4df0ae1a4078428b1375'} {{(pid=81459) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.239505 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:17.246067 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1477/2947] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Sep 7 16:14:17 2022] DELETE /identity/v3/users/690176f79d9d4df0ae1a4078428b1375 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:14:17.411298 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Authenticating user token {{(pid=81458) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:14:17.414071 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.fernet_utils [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81458) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:14:17.428812 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b03a3-97ab-4dc9-9fb9-79a78ce5061d admin admin] Validating token access rules against request {{(pid=81458) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:14:17.480514 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:17.487999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '91fe14f693174ef291042aa145475ad4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4102cbba3e844357b14c2a5139b42e69', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81458) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:14:17.488636 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:14:17.488981 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:14:17.489418 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] PATH_INFO: `/v3/projects/e821597a56454269a77d3c529a03a2cf` {{(pid=81458) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:14:17.490764 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e821597a56454269a77d3c529a03a2cf)` {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:14:17.492694 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] RBAC: Authorization granted {{(pid=81458) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:14:17.537612 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e821597a56454269a77d3c529a03a2cf', 'request_id': 'req-3316c956-f30a-416f-9488-337ec72b0904'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.545197 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e821597a56454269a77d3c529a03a2cf', 'request_id': 'req-3316c956-f30a-416f-9488-337ec72b0904'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.550771 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e821597a56454269a77d3c529a03a2cf', 'request_id': 'req-3316c956-f30a-416f-9488-337ec72b0904'} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.561591 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] The token cache is being invalidate because project e821597a56454269a77d3c529a03a2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81458) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 07 16:14:17.561897 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.562509 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG keystone.notifications [None req-3316c956-f30a-416f-9488-337ec72b0904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81458) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 07 16:14:17.563999 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: [pid: 81458|app: 0|req: 1471/2948] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Sep 7 16:14:17 2022] DELETE /identity/v3/projects/e821597a56454269a77d3c529a03a2cf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 07 16:14:27.248766 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=1 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:14:27.558580 dv-cinder-853315-8-iscsi devstack@keystone.service[81458]: DEBUG dbcounter [-] [81458] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81458) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 07 16:15:14.374943 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Authenticating user token {{(pid=81459) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 07 16:15:14.376380 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:14.482776 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f5419-7e32-4c49-8135-dc3d2ff572f4 admin admin] Validating token access rules against request {{(pid=81459) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 07 16:15:14.540576 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc95332f41224ecdb42604ac98262545', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd73a0e7260464132bcc1491e2ebb347d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81459) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 07 16:15:14.541189 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] REQUEST_METHOD: `GET` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 07 16:15:14.541475 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] SCRIPT_NAME: `/identity` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 07 16:15:14.541749 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81459) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 07 16:15:14.543690 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.fernet_utils [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81459) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 07 16:15:14.555513 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 07 16:15:14.557145 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5cc0c5-3256-4f18-88dd-43d89d2cbebe service placement] RBAC: Authorization granted {{(pid=81459) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 07 16:15:14.566804 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: [pid: 81459|app: 0|req: 1478/2949] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Sep 7 16:15:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 07 16:15:24.562673 dv-cinder-853315-8-iscsi devstack@keystone.service[81459]: DEBUG dbcounter [-] [81459] Writing DB stats keystone:SELECT=74 {{(pid=81459) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}